I would like to test OpenGamma with actual data coming from an upstream system (where we store trades, mkt data etc…), instead of sample data provided in the install.
In such a scenario, how would you suggest we handle data to be processed ? I’m thinking of an in-memory datagrid, updated from activemq.
You’ve basically got several options for non-live data (security reference, time series, portfolio/position data, holidays, etc):
ETL - have a process that regularly dumps a full dataset from your source system and imports into the OpenGamma masters. This has the advantage of being relatively easy and decoupled from OpenGamma specifics, but obviously is less desirable for real-time operation
Incremental ETL/Triggers/JMS - use triggers or message bus to carry inserts/updates etc. from your source system to OpenGamma data store implementations. This has the advantage of keeping the two systems in sync all the time and also allows the use of the Bi-temporal versioning in the OpenGamma data stores that might not be available in the source system. It also allows the use of the existing OpenGamma analytics library
Custom Master/Source implementations. You build your own custom implementations of the Master and/or Source interfaces that directly pull data from your source system and use your own security schemas separate from those in OG-Financial. This has the advantage that you can model your securities and positions/trades however you want, but the disadvantage that the out-of-the-box analytics integration will not work unless you've stuck to the same *Security interfaces as used in OG-Financial.
You sound like you're homing in on number 2, which I think sounds completely reasonable. Note you can use our (pretty simple) InMemory*Master implementations if you wanted to do that here, or you could provide your own implementation of the interfaces or customisations of ours.
For live market data, if you’re pulling from a system other than Bloomberg (or one of our commercially supported types), you’ll need to write a subclass of AbstractLiveDataServer. That’s usually pretty easy as most of the work tracking subscriptions and so on is already done for you in that abstract base class. We’ve currently got that working against Bloomberg (Desktop, SAPI/Managed B-PIPE), Activ Financial, Tullet Prebon (direct feed), ICAP (direct feed) and Thomson Reuters so we’re pretty confident it’ll be possible to hook it up easily to other data providers.
Helping with these tasks is exactly the type of thing we typically do as part of a commercial engagement, so please do get in touch if you want to go down that path.