OG-2.x how to configure HDFS as a storage?


#1

Hi,
I want to store the incoming data into the HDFS.
Can any one plz advise what are the code changes in OpenGamma Source code I will have to make ?
Thanks,
Abhi


#2

We have not explored storing data in HDFS, so cannot give much advice on how easy it will be.

If you are looking to store data such as trades, securities and configuration, then you need to implement either the Source interfaces or the Master interfaces. You implement the Source interfaces if you just want to provide read-only data to the system. You implement the Master interfaces if you want to allow read-write updates. It is much easier to go with the source level.

See the interfaces SecuritySource and SecurityMaster for example code to implement. Once created, you have to write a ComponentFactory similar to DbSecurityMasterComponentFactory to configure your code in.


#3

Thanks for help Stephen. I will explore these APIs.


#4

Let me know if you need any help.

I’m working on python integration with opengamma. What I’ve done is use python to download bitcoin data, and then convert it to HDF5, and it would be interesting to load it into java.

I’ve also been able to get python to communicate with OpenGamma via py4j. I’ll talk about this in another thread.