In batch mode , is the dependency graph build for the entire batch or for each cycle


#1

Hi ,

My understanding on the dependency graph is that for each cycle of execution of viewDefenition over portfolio , a dependency graph is build based on the function names and constrains given in viewDefention.
And then value is calculated by traversing this graph. Here the values on each node is also cached and duplicate execution of a function is thus averted.
In case of batch mode , where we run the cycle over many dates , are the PRIMARY types (of same constrains) values re-computed for each run ?
Or is the graph initially made for the entire batch jobs rather than just a cycle ?

Thanks
Vineeth


#2

Firstly, strictly speaking, the graph itself isn’t directly executed, but used to build a queue of jobs which are then dispatched to the calc node.

With regard to the scope of graph building, no, the graph is made for the initial cycle and then re-analysed for each subsequent cycle.

What happens in the current build is that if anything has changed between cycles that means the previous graph is not still valid, it rebuilds the dependency graph in it’s entirety.

However, the work to improve this situation, and slightly alter some of the function interface is currently waiting on a branch for integration once we’ve pushed out the 1.2 release. This will:

  • Better track portfolio changes, and they will themselves automatically trigger dependency graph rebuilds.
  • Remove the need for ID combining inside the PositionSource by removing the requirement that Positions need to know which portfolio nodes they belong to. We found alternative solutions to the scenarios we originally thought this would be required for (e.g. computing component VaR).
  • Make the target type concept much more flexible by replacing the current PRIMITIVE/SECURITY/POSITION/TRADE/PORTFOLIO_NODE target types with a class-based system. This will improve flexibility, make the engine more efficient at building the dependency graph (because it can more quickly find the functions necessary) and allow the use of custom types of target.
  • Perform incremental rebuilds of the dependency graph only rebuilding what's required between cycles. This should lead to a pretty substantial performance increase when running scenarios and make everything faster and smoother.
Work is also ongoing on the ability to spawn sub-view processes from an engine function. This will allow, for example, a single view cycle to compute e.g. full reprice VaR or initiate a scenario itself.

#3

Hello @jim ,

The last point to “Perform incremental” was what i was looking for. This will bring out a lot better performance. Hoping to see it soon :slight_smile:


#4

Hi Jim,
I am very interested in the class based target types feature that you mention here .
Is this available now that 1.2 is available. If so I will take a look.
Thanks,
Neil


#5

Hi Neil,
It’s wasn’t available in 1.2.x, but it is available on the develop branch along with incremental dependency graph compilation and nested view clients. It literally finally got merged in within the last day or two.
Jim