View dependency graph

OG-Platform
1

For the purpose of debugging , is there a way i can view or see the dependencies in the dependency graph created by OG ? This should include function names , its class , its value requirements and results.

Thanks
Vineeth

2

If the graph successfully builds, you can see it by (in the old UI), left-clicking on the cell in question. If you want to see the paths the engine takes trying to satisfy a requirement (but failing), you can change:

DependencyGraphBuilder.java:71

System.getProperty("DependencyGraphBuilder.dumpDependencyGraph", "FALSE").equalsIgnoreCase("TRUE");

becomes:

System.getProperty("DependencyGraphBuilder.dumpDependencyGraph", "TRUE").equalsIgnoreCase("TRUE");

do the same on line 77. Then in DependencyGraphBuilderFactory:52 change

private boolean _enableFailureReporting = System.getProperty("DependencyGraphBuilderFactory.enableFailureReporting", "FALSE").equalsIgnoreCase("TRUE");

to

private boolean _enableFailureReporting = System.getProperty("DependencyGraphBuilderFactory.enableFailureReporting", "TRUE").equalsIgnoreCase("TRUE");

This will then dump out files called resolutionFailureX.txt where X=1,2,3,4 etc. You’ll find this file in wherever java.io.tmp is pointed to, which can be a strange place. Easiest way to find it is with a ‘find’ command e.g.

sudo find / -name resolutionFailure1.txt
3

Hello @jim ,

Is it possible to see the entire graph. That is i would like to see what all are the registered functions and what are its prerequisite

Thanks
Vineeth

4

Not sure what you’re trying to achieve. You might be able to dump the whole graph by changing some more if the flags in DependecyGraphBuilder to true (I can’t check the source code from my current location) but you’ll get a very large dump.

5

Hello @jim ,

I am trying to register a function and somehow the dependency graph is not able to resolve my function when a viewDefention is executed. What i wanted to see is that if my function is successfully registered and some possible clues on how my function got missed.

Thanks
Vineeth

6

Ok, well your problem is that (assuming you’re not getting any output value as opposed to getting the value you want, but from a different function) your function isn’t in the dependency graph, so you probably won’t learn a huge amount by looking at it. The more interesting output will be from the failure report I mentioned how to generate above as this will show the different function combinations the engine is trying before giving up. The approach we tend to take is:

Create a portfolio with a single relevant position if it’s an output that is on a security, position or position node (you can create a portfolio-less view definition if your output is a primitive level output). Then create a view definition with just the single output you want on the asset class you’re looking it. Then run the view and see how the engine tries to build the dependency graph.

Typical problems are:

  • Your function needs to operate on the correct target level - if you are calculating something that is requires full trade/position level data, return POSITION or TRADE level in getTargetType(). If you're operating at SECURITY level, then you're relying on the existence of using a standard PositionScalingFunction - you'll need to make sure you register a position scaling function for the value requirement name you're using in whichever function repository you're using (e.g. DemoStandardFunctionRepository). If you also want aggregation, to the PORTFOLIO_NODE level and it's a simple summing operation, you can add a standard summing function in the same way. You will of course need to make sure that the standard summing/scaling functions can handle the type of value you're outputting, but they can generally handle most types.
  • If you are passing constraint parameters to your function, your simpler getResults need to have specified that you can accept those constraints, typically with a ValueProperties builder passed e.g. .withAny("Param1"). This basically says 'my function requires this constraint, but I will accept any value. Now you don't need to have a second getResults (there are two - we're planning to rename them), which gets called after the engine has resolved to use the function, and provides more information about the target requirements. Think of the simpler getResults as the engine asking 'tell me in broad terms what you can produce', and the second getResults (with the extra parameters) is 'ok, here is what the user actually asked for, do you want to make your reply more specific given you know more now'.
  • You need to make sure that any applicable constraints are passed on to the input requirements e.g. often you will need to pass the curve configuration name, possibly the currency and so on to the underlying e.g. yield curve requirements if you have any.
A good way of seeing what's going on is by setting breakpoints in all the methods called by the engine and seeing what it's doing and stepping through. Also, make sure you up the logging level so you can see what the engine is up to.

Hope that helps.

Jim

7

Just to elaborate a bit on this process: we are working on several projects to make this all easier. One is that we’ll be providing a DSL-like language to effectively automatically build the getRequirements, getResults and so on, without so much boiler-plate. You just specify your inputs and outputs and any relationship between them. It’ll be pure Java, but the API will let you write it more like a constraint scripting language. We’re also planning to add various utilities and so on to make the most common tasks simpler.

This work is underway right now, but will take probably 6-8 weeks to be completed.

The second big improvement is that we’re working on routing logging messages directly to the UI. This will mean that output cells will show little marks if they have WARN or ERROR log messages, and you’ll be able to see any Java log output just from that value, rather than having to trawl through the whole log. This will largely remove the necessity to create a separate view with a single value/position in it for debugging.

We’ve already created the back-end part of the logging improvements, and it’ll be available in the next 3-4 weeks I suspect.

8

Hello @jim ,

Thanks for the detailed explanation.
I finally got hold on the adding new functions to the dependency graph.
Earlier i ignored the primitive type and couldn’t see the full picture without it.

Also in your earlier comment , you had mentioned about DSL like language. Can you kindly elaborate what exactly is it for and what kind of language it is ?

We were thinking of adding a scripting langauge like R or something to define the execution body of a function. We would like to know if this is anything close.

Thanks
Vineeth

9

It’s actually a Java-based ‘Internal DSL’ in the parlance of Martin Fowler (Domain Specific Languages, Addison Wesley). This means you use a technique like method chaining to be able to write Java that looks like a DSL:

void setup() {
  RequirementsBuilder
    .outputs()
      .output(HistoricalVaR)
        .constraint(Currency, "@currency")
        .constraint(LookBackYears, "@lookback")
    .inputs()
      .input(YieldCurve)
        .constraint(CurveCurrency, "@currency")
        .constraint(Currency, "@currency")
      .inputs(PnLSeries)
        .constraint(Currency, "@currency")
        .constraint(Period, "@lookback")
        .constraint(SamplingFrequency, "@samplingFrequency")
    .constraints()
       .eq("@samplingFrequency", "Weekly")
       // by using @currency and @lookback in inputs and outputs, 
       // there's an implicit equals there.
  .build();
}

Not a complete example, but hopefully you get the idea. We might also produce an external non-java version of the DSL for when coding functions in external languages like R, Matlab or C++, but we haven’t done anything in that direction yet.