CPU at 99% with ExampleLiveDataServer before first job

I’m running the example OG deployment, v2.1, in a distributed setup with remote PostgreSQL and ActiveMQ. Each of the OG servers seems to run at 99% CPU until it gets a calc job.

jstack shows that it is running ExampleLiveDataServer.SimulatedMarketDataJob.runOneCycle which appears to be chewing up one core. (Full stack trace below [1].)

Curiously as soon as I send a calc job to the node (click “Load” in the GUI with some calculation) this thread stops and CPU runs at a manageable ~10% per job and down to ~2% when the job is killed.

So it’s not a problem – the node settles down when he has too. But it’s odd…


[1] The active thread:

“ExampleLiveDataServer-1” daemon prio=10 tid=0x00007f36e158e000 nid=0xa22 runnable [0x00007f36c42c4000]
java.lang.Thread.State: RUNNABLE
at java.util.AbstractCollection.addAll(AbstractCollection.java:341)
at java.util.HashSet.(HashSet.java:117)
at com.opengamma.livedata.server.StandardLiveDataServer.getSubscriptions(StandardLiveDataServer.java:1065)
at com.opengamma.livedata.server.StandardLiveDataServer.getActiveSubscriptionIds(StandardLiveDataServer.java:1041)
at com.opengamma.examples.simulated.livedata.ExampleLiveDataServer$SimulatedMarketDataJob.runOneCycle(ExampleLiveDataServer.java:259)
at com.opengamma.util.TerminatableJob.run(TerminatableJob.java:43)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)

This is caused by a fault in the simulated market data server code when it is running with no active subscriptions. This was fixed in the 2.2.0-M5 release.

I’ve had better experiences running against the 2.2 prerelease than 2.1. There have been a lot of fixes and enhancements in the 2.2 prerelease than with 2.1.

The only problem that I have is that there have been times when someone has changed the code of 2.2 and not backported it to the example server, so I work against a checkpoint, and update that every week or so.