Example server with Postgres

OG-Platform
1

Hi,
Having failed to build from Github, I got the latest source package from the download section on the website. This all works ( nice web ui, btw! very smooth ). I then tried to upgrade the install from using HSQL to Postgres using a single schema. After a little while, ( took me a little while to work out that there’s no schema needed with the example installation + default configuration). The changes, so far are to the demoMasters-example.properties file to look like:

opengamma.config.jdbc.driver=org.postgresql.Driver
#opengamma.config.jdbc.url=jdbc:postgresql://localhost:5432
opengamma.config.jdbc.url=jdbc:postgresql:og_financial
opengamma.config.jdbc.username=postgres
opengamma.config.jdbc.password=
opengamma.config.db.dbhelper=com.opengamma.util.db.PostgreSQLDbHelper

opengamma.financial.jdbc.driver=org.postgresql.Driver
opengamma.financial.jdbc.url=jdbc:postgresql:og_financial
opengamma.financial.jdbc.username=postgres
opengamma.financial.db.dbhelper=com.opengamma.util.db.PostgreSQLDbHelper

opengamma.hts.jdbc.driver=org.postgresql.Driver
opengamma.hts.jdbc.url=jdbc:postgresql:og_financial
opengamma.hts.jdbc.username=postgres
opengamma.hts.db.dbhelper=com.opengamma.util.db.PostgreSQLDbHelper

opengamma.overnight-batch.jdbc.driver=org.postgresql.Driver
opengamma.overnight-batch.jdbc.url=jdbc:postgresql:og_financial
opengamma.overnight-batch.jdbc.username=postgres
opengamma.overnight-batch.db.dbhelper=com.opengamma.util.db.PostgreSQLDbHelper

After running the ant demo-database, I can see some data in the cfd_config table, however, the ant process is failing with the following error:
[java] org.springframework.beans.factory.BeanCreationException: Error creating bean with name ‘curveLoader’ defined in class path resource [com/opengamma/financ
ial/demoMasters.xml]: Instantiation of bean failed; nested exception is org.springframework.beans.BeanInstantiationException: Could not instantiate bean class [com.o
pengamma.financial.analytics.ircurve.YieldCurveConfigPopulator]: Constructor threw exception; nested exception is org.fudgemsg.wire.FudgeRuntimeIOException
[java] at org.springframework.beans.factory.support.ConstructorResolver.autowireConstructor(ConstructorResolver.java:288)
[java] at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.autowireConstructor(AbstractAutowireCapableBeanFactory.java:1003)
[java] at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:907)
[java] at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:485)
[java] at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:456)
[java] at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:291)
[java] at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:222)
[java] at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:288)
[java] at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:190)
[java] at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:281)
[java] at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:190)
[java] at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:580)
[java] at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:895)
[java] at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:425)
[java] at org.springframework.context.support.ClassPathXmlApplicationContext.(ClassPathXmlApplicationContext.java:139)
[java] at org.springframework.context.support.ClassPathXmlApplicationContext.(ClassPathXmlApplicationContext.java:83)
[java] at com.opengamma.examples.loader.DemoDatabasePopulater.main(DemoDatabasePopulater.java:76)
[java] Caused by: org.springframework.beans.BeanInstantiationException: Could not instantiate bean class [com.opengamma.financial.analytics.ircurve.YieldCurveCo
nfigPopulator]: Constructor threw exception; nested exception is org.fudgemsg.wire.FudgeRuntimeIOException
[java] at org.springframework.beans.BeanUtils.instantiateClass(BeanUtils.java:141)
[java] at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:108)
[java] at org.springframework.beans.factory.support.ConstructorResolver.autowireConstructor(ConstructorResolver.java:280)
[java] … 16 more
[java] Caused by: org.fudgemsg.wire.FudgeRuntimeIOException
[java] at org.fudgemsg.wire.FudgeDataInputStreamReader.readFieldValue(FudgeDataInputStreamReader.java:362)
[java] at org.fudgemsg.wire.FudgeDataInputStreamReader.consumeFieldData(FudgeDataInputStreamReader.java:322)
[java] at org.fudgemsg.wire.FudgeDataInputStreamReader.next(FudgeDataInputStreamReader.java:194)

Does anyone have any advice on where to start trouble shooting this ?
Thanks in advance,

2

Hi,

I’m getting one of my colleagues, Kevin, to look into this and try to reproduce the problem. He should get back to you here some time tomorrow with an update.

Jim

3

Thanks very much Jim. Sorry - forgot to add that it is the latest version of Postgres and the patch_20 schema. Please let me know if you need anything else.

4

Hello,

Thanks for the query. I am trying to reproduce the issue and will get back to you as soon as I have something to add.

Regards,
Kevin

5

Hi,

The behaviour you observed is due to the fact that our 0.9 release expects a pre-version 9 Postgres. Postgres 9 introduced a new bytea (binary blob) output format which is not supported by the Postgres 8.4 JDBC connector used in our 0.9 release. Naturally, our 1.0 release will use a newer Postgres JDBC connector, which supports both old and new bytea output formats.

For the time being, there are a couple of ways to get around this without too much effort, both outlined in http://jira.opengamma.com/browse/PLAT-1545 and reproduced below. I have successfully used your demoMasters-example.properties in conjunction with the second solution. Note that you will probably need to drop the existing og_financial database and recreate it in order to clean it out, unless you start from scratch.

Solution 1: (quick and dirty) Make the Postgres server output the old (pre-9) format by inserting the following line in postgresql.conf

bytea_output=‘escape’

Solution 2: (requires the source release) Upgrade the postgres-jdbc driver to a version >= 9, which accepts the new format, by modifying ivy.xml in OG-Utils and including the new version in our ivy repos. Here is the correct dependency in projects/OG-Utils/ivy.xml to get postgresql-jdbc to work with Postgres 9

Apologies for the inconvenience. Please do let me know how this works out for you!

Regards
Kevin

6

Strange - I already had the ivy entry as you have described in step 2. Hacking the db as per step 1 have fixed the problem. Once again, thanks for your help

7

Where can I find help to run OG-Examples completely on Postgresql database?

I followed the link http://docs.opengamma.com/display/DOC071/Running+the+Example+Engine
but couldnot find ‘demoIntegrationMasters-shareddev.properties’ in OG-Examples/config directory.

8

That document is for version 0.7.1 and we’ve significantly changed things since then. Having said that, I can see why you ended up there, as there isn’t much information about how to modify the examples. The file you’re looking for is in OG-Examples/config/fullstack/example.properties. This file is part of a chain from example-*.properties (which one depends on whether you’re running tests, creating a binary package or running in development mode) through to the actual component configuration file example.ini in the same directory. I’ll open a JIRA to improve that aspect of the documentation in the meantime.

9

JIRA issue is PLAT-2162 and is assigned.

10

Let me know if this page is sufficient
http://docs.opengamma.com/display/DOC/Changing+configuration+for+other+databases

11

Thanks Jim, That was pretty easy to run the server on postgresql with out any error.
but I think the exampleDabasePopulator is not using the same context which the server is using. I am new to spring framework. I found that the populator still loads the data to hsqldb.

12

Sorry, I forgot to mention that there’s a similar file that needs editing under OG-Examples/config/toolcontext. This is the configuration used for the database populator tool. It’s not actually Spring - we got fed up with spring files being so complex and error prone so we wrote our own distributed component system that’s much simpler. It can contain spring contexts as components, and we use them like that sometimes. See the documentation for more details.

13

Editing toolcontext/examples.properties file solved the problem.
Thank you Jim. Thank you Stephen for updated documentation.