I have been trying to insert timeseries data points from a csv file which contain values of a number of datafields. Then I noticed that the htsmaster will not allow insertion of datapoints prior to the first date that exists in the timeseries table for the same datafield. This compels the timeseries input always to keep the chronological order. is there anyway to overcome this limitation?
It’s probably because you’re using add rather than update. The hts master has some semantics that can seem counter-intuitive, but that revolve around providing an efficient way to store versions and corrections. The idea is that for any given time series an hts operation is a whole series operation. Thus if you try to add data points before the latest point then you are changing values in the past (going from no value to a value). This requires you to perform an update (or correction - a correction would be used for changing an existing data point that you want to appear updated after a different point in time than today - e.g, if you run a report from a year ago do you want the original value or the updated value) instead of an add. UPDATE: A correction is used to modify a value that’s already present in the time series. To try and make all this easier to use we’ve written a helper class called HistoricalTimeSeriesWriter (I’ll check the name when I’m at my desk) that translates a simple addOrUpdate UPDATE: method called writeTimeSeries() into the appropriate underlying calls.
We should probably either change the behaviour or at the very least document the behaviour more completely.
I hope that helps.
Jim
Just to confirm, the class I mentioned is: com.opengamma.master.historicaltimeseries.impl.HistoricalTimeSeriesWriter. From the JavaDocs, it’s actually not clear what happens when you call writeTimeSeries with a time series containing e.g. just one data point, but looking at the code, it does what you’d want. If you’re updating existing points then it performs a correction, if it’s new data points prior to the latest in the series, it does an update, and if it’s an existing data point then it does a correction. If the series contains values that cross these boundaries it will split them up into a sequence of operations: e.g. add the new values, update any older or missing values and correct any that are different. As I alluded to above, we really need to review the API - it’s largely the way it is for consistency with other master APIs, but it’s definitely very low level, and not very intuitive. I’ll update the JavaDocs now and add a pointer from the HTSMaster interface to the writer.
Thank you jim for the reply.
I forgot to give the hint that I have been using add function and the timeseries contain only one value when trying to insert. Structure of the csv file is basically two dimensional ie. a single row contain values of many datafields. so practically I had to process it as single value timeseries. and I could add values which keep the chronology in the csv file and successfully retrieved back as s single timeseries.
That’s such a common case that we should probably add a method for just updating a single data point, but you should be able to use the HTSWriter class I mentioned with just the single data point in the time series and get the same effect.