Simple Zero Curve Calibration (Deposit+Futures+Swap)

Thank you for the release 1.0! It seems more stable and fast.

Unfortunately the zero curve calibration is still somewhat inaccurate. Given a set of deposits (till 8M + stub), a set of six following futures and then swaps to 60Y, only the deposit part has a mean error of E-14. From the first future, the error becomes instantly E-5 and even worse.

Since the benchmark curve is point-based, I’m directly comparing Strata Curve “parameters” with benchmark points (linear interpolation for both). I also tried comparing the discount factors at same dates but the problem remains.

The curve settings are:

EUR-STD,Zero,Act/365F,Linear,Flat,Flat

The group settings are:

EUR-Standard,Discount,EUR,EUR-STD
EUR-Standard,Forward,EUR-EURIBOR-3M,EUR-STD
EUR-Standard,Forward,EUR-EURIBOR-6M,EUR-STD

The calibration nodes are:

EUR-STD,ON,OG-Ticker,EUR-ON,MarketValue,DEP,EUR-ShortDeposit-T0,1D,
EUR-STD,TN,OG-Ticker,EUR-TN,MarketValue,DEP,EUR-ShortDeposit-T1,1D,
EUR-STD,SN,OG-Ticker,EUR-SN,MarketValue,DEP,EUR-ShortDeposit-T2,1D,
EUR-STD,1W,OG-Ticker,EUR-1W,MarketValue,DEP,EUR-ShortDeposit-T2,1W,
EUR-STD,2W,OG-Ticker,EUR-2W,MarketValue,DEP,EUR-ShortDeposit-T2,2W,
EUR-STD,3W,OG-Ticker,EUR-3W,MarketValue,DEP,EUR-ShortDeposit-T2,3W,
EUR-STD,1M,OG-Ticker,EUR-1M,MarketValue,DEP,EUR-Deposit-T2,1M,
EUR-STD,2M,OG-Ticker,EUR-2M,MarketValue,DEP,EUR-Deposit-T2,2M,
EUR-STD,3M,OG-Ticker,EUR-3M,MarketValue,DEP,EUR-Deposit-T2,3M,
EUR-STD,4M,OG-Ticker,EUR-4M,MarketValue,DEP,EUR-Deposit-T2,4M,
EUR-STD,5M,OG-Ticker,EUR-5M,MarketValue,DEP,EUR-Deposit-T2,5M,
EUR-STD,6M,OG-Ticker,EUR-6M,MarketValue,DEP,EUR-Deposit-T2,6M,
EUR-STD,7M,OG-Ticker,EUR-7M,MarketValue,DEP,EUR-Deposit-T2,7M,
EUR-STD,8M,OG-Ticker,EUR-8M,MarketValue,DEP,EUR-Deposit-T2,8M,
EUR-STD,STUB,OG-Ticker,STUB,MarketValue,DEP,EUR-Deposit-T2,P8M4D,
EUR-STD,Mar17,OG-Future,Mar17,SettlementPrice,IFU,EUR-EURIBOR-3M-Quarterly-IMM,7D+3,
EUR-STD,Jun17,OG-Future,Jun17,SettlementPrice,IFU,EUR-EURIBOR-3M-Quarterly-IMM,7D+4,
EUR-STD,Sep17,OG-Future,Sep17,SettlementPrice,IFU,EUR-EURIBOR-3M-Quarterly-IMM,7D+5,
EUR-STD,Dec17,OG-Future,Dec17,SettlementPrice,IFU,EUR-EURIBOR-3M-Quarterly-IMM,7D+6,
EUR-STD,Mar18,OG-Future,Mar18,SettlementPrice,IFU,EUR-EURIBOR-3M-Quarterly-IMM,7D+7,
EUR-STD,Jun18,OG-Future,Jun18,SettlementPrice,IFU,EUR-EURIBOR-3M-Quarterly-IMM,7D+8,
EUR-STD,3Y,OG-Ticker,EUR-IRS6M-3Y,MarketValue,IRS,EUR-FIXED-1Y-EURIBOR-6M,3Y,
EUR-STD,4Y,OG-Ticker,EUR-IRS6M-4Y,MarketValue,IRS,EUR-FIXED-1Y-EURIBOR-6M,4Y,
EUR-STD,5Y,OG-Ticker,EUR-IRS6M-5Y,MarketValue,IRS,EUR-FIXED-1Y-EURIBOR-6M,5Y,
EUR-STD,6Y,OG-Ticker,EUR-IRS6M-6Y,MarketValue,IRS,EUR-FIXED-1Y-EURIBOR-6M,6Y,
… and so on

Quotes are:

2016-07-07,OG-Ticker,EUR-ON,MarketValue,-0.003699999
2016-07-07,OG-Ticker,EUR-TN,MarketValue,-0.0037
2016-07-07,OG-Ticker,EUR-SN,MarketValue,-0.0037
2016-07-07,OG-Ticker,EUR-1W,MarketValue,-0.0037
2016-07-07,OG-Ticker,EUR-2W,MarketValue,-0.0037
2016-07-07,OG-Ticker,EUR-3W,MarketValue,-0.0037
2016-07-07,OG-Ticker,EUR-1M,MarketValue,-0.0036
2016-07-07,OG-Ticker,EUR-2M,MarketValue,-0.0032
2016-07-07,OG-Ticker,EUR-3M,MarketValue,-0.0029
2016-07-07,OG-Ticker,EUR-4M,MarketValue,-0.0025
2016-07-07,OG-Ticker,EUR-5M,MarketValue,-0.0022
2016-07-07,OG-Ticker,EUR-6M,MarketValue,-0.0018
2016-07-07,OG-Ticker,EUR-7M,MarketValue,-0.0016
2016-07-07,OG-Ticker,EUR-8M,MarketValue,-0.0014
2016-07-07,OG-Ticker,STUB,MarketValue,-0.0013862069
2016-07-07,OG-Future,Mar17,SettlementPrice,1.003825
2016-07-07,OG-Future,Jun17,SettlementPrice,1.003975
2016-07-07,OG-Future,Sep17,SettlementPrice,1.004025
2016-07-07,OG-Future,Dec17,SettlementPrice,1.004125
2016-07-07,OG-Future,Mar18,SettlementPrice,1.004125
2016-07-07,OG-Future,Jun18,SettlementPrice,1.004125
2016-07-07,OG-Ticker,EUR-IRS6M-3Y,MarketValue,-0.00239
2016-07-07,OG-Ticker,EUR-IRS6M-4Y,MarketValue,-0.00215
2016-07-07,OG-Ticker,EUR-IRS6M-5Y,MarketValue,-0.00165
2016-07-07,OG-Ticker,EUR-IRS6M-6Y,MarketValue,-0.00092
… and so on

Maybe I am doing something wrong, but I followed the Calibration Examples in Strata source.

Thank you for your time. I hope that together we will find out the problem.


EDIT: to be thorough, benchmark quotes (cut at 6Y) are

-0.003751407
-0.003751437
-0.003751431
-0.003751492
-0.003751609
-0.003751737
-0.003662094
-0.003275567
-0.002975121
-0.002574093
-0.002270083
-0.00186681
-0.001661529
-0.001457572
-0.001443504
-0.00209635
-0.002520221
-0.002784454
-0.002987352
-0.003143395
-0.003261472
-0.002383656
-0.002145337
-0.001648484
-0.0009204

Thanks for the example. The config files indicate that you are calibrating to create a single curve of zero-rates that is used for three purposes - discounting, EURIBOR 3M and EURIBOR 6M.

Firstly, the calibrations csv file has been enhanced to allow either relative futures or absolute futures. As such, there is no need to use the 7D+3 notation now. The following 6 lines can replace those in your file:

EUR-STD,Mar17,OG-Future,Mar17,SettlementPrice,IFU,EUR-EURIBOR-3M-Quarterly-IMM,Mar17,
EUR-STD,Jun17,OG-Future,Jun17,SettlementPrice,IFU,EUR-EURIBOR-3M-Quarterly-IMM,Jun17,
EUR-STD,Sep17,OG-Future,Sep17,SettlementPrice,IFU,EUR-EURIBOR-3M-Quarterly-IMM,Sep17,
EUR-STD,Dec17,OG-Future,Dec17,SettlementPrice,IFU,EUR-EURIBOR-3M-Quarterly-IMM,Dec17,
EUR-STD,Mar18,OG-Future,Mar18,SettlementPrice,IFU,EUR-EURIBOR-3M-Quarterly-IMM,Mar18,
EUR-STD,Jun18,OG-Future,Jun18,SettlementPrice,IFU,EUR-EURIBOR-3M-Quarterly-IMM,Jun18,

(These will make no difference to pricing, but are a little clearer)

I have run the CalibrationCheckExample with your data and observed your results (I think). The y-values of the calibrated curve from ON to 8M match closely to the numbers you added, but for the futures and swaps the gap is bigger. The question is why the results differ.

What we can say is that the Strata results are self-consistent. The CalibrationCheckExample does two things - first it calibrates the curve, then it checks that when the trades are priced using the curve the resulting PV is zero. The PV is zero to around 1e-15 in general.

Since the results differ based on asset class, it is possible that the pricing methodology is different between the benchmark system and Strata. Strata will use a simple discounting method for the futures and swaps (no convexity adjustment). Does the benchmark?

Another possibility is how the two systems decide on the date associated with the node. Strata uses the “end date” of the trade by default, which for a future is the implied maturity date and for the swap is the last date of the last period.

However, our quant Marc thinks that the difference may be due to how the period on the forward curve is determined for futures. Strata uses the implied period related to the index, which is 3 months in this case. Thus, Strata calculates that the Mar17 future has a fixing on the 2017-03-13, a start date of 2017-03-15 and an end date of 2017-06-15. We are aware that some systems use an end date of the 3rd Wednesday ie. 2017-06-21. Clearly, if your benchmark system does this, your results will be different. Can you determine if this is the case?

Thank you very much for the clear explanation.

After some investigation I found that a convexity spread is actually applied on 4 futures. Inserting the correct values now restores the error on futures below 1e-10, which is very good.
Updated calibration nodes are the following:

EUR-STD,Mar17,OG-Future,Mar17,SettlementPrice,IFU,EUR-EURIBOR-3M-Quarterly-IMM,Mar17,-0.00025
EUR-STD,Jun17,OG-Future,Jun17,SettlementPrice,IFU,EUR-EURIBOR-3M-Quarterly-IMM,Jun17,-0.00025
EUR-STD,Sep17,OG-Future,Sep17,SettlementPrice,IFU,EUR-EURIBOR-3M-Quarterly-IMM,Sep17,
EUR-STD,Dec17,OG-Future,Dec17,SettlementPrice,IFU,EUR-EURIBOR-3M-Quarterly-IMM,Dec17,
EUR-STD,Mar18,OG-Future,Mar18,SettlementPrice,IFU,EUR-EURIBOR-3M-Quarterly-IMM,Mar18,0.00025
EUR-STD,Jun18,OG-Future,Jun18,SettlementPrice,IFU,EUR-EURIBOR-3M-Quarterly-IMM,Jun18,0.0005

The error on swaps still remains very high (1e-4).

Fixing, start and end date are correct for futures and the “end dates” associated with the nodes are all correct.

Great to here that your futures are matching better (although I tried your config and it didn’t improve the match for me).

On the swaps, I don’t think we have anything to add beyond the ideas in the last reply. Strata is using straight discounting, no convexity for example. It may be necessary for you to drill down into one of the swaps and see if the generated periods and values match to find the problem. In Strata, the “explain present value” measure can be used to drill down into a swap.

About futures, you are right. I forgot a couple of zeroes in the decimals. The correct entries are the following:

EUR-STD,Mar17,OG-Future,Mar17,SettlementPrice,IFU,EUR-EURIBOR-3M-Quarterly-IMM,Mar17,-0.0000025
EUR-STD,Jun17,OG-Future,Jun17,SettlementPrice,IFU,EUR-EURIBOR-3M-Quarterly-IMM,Jun17,-0.0000025
EUR-STD,Sep17,OG-Future,Sep17,SettlementPrice,IFU,EUR-EURIBOR-3M-Quarterly-IMM,Sep17,
EUR-STD,Dec17,OG-Future,Dec17,SettlementPrice,IFU,EUR-EURIBOR-3M-Quarterly-IMM,Dec17,
EUR-STD,Mar18,OG-Future,Mar18,SettlementPrice,IFU,EUR-EURIBOR-3M-Quarterly-IMM,Mar18,0.0000025
EUR-STD,Jun18,OG-Future,Jun18,SettlementPrice,IFU,EUR-EURIBOR-3M-Quarterly-IMM,Jun18,0.000005

Now the error jump between last future and first swap is even more big.

I’ll try to drill down into the swaps generated during calibration. Is there a Report template for EXPLAIN_PRESENT_VALUE?

No, there is no report template for EXPLAIN_PRESENT_VALUE. You need to add the column to

List<Column> columns = ImmutableList.of(
    Column.of(Measures.PRESENT_VALUE,
    Column.of(Measures.EXPLAIN_PRESENT_VALUE));

and then when you get the Results object, simply query it:

Results results = ...
ExplainMap explain = results.get(indexOfSwapTrade, ColumnName.of(Measures.EXPLAIN_PRESENT_VALUE.getName()));

Digging into Strata swaps after calibration, the only differences I found are on fixing rate

-0.00180 strata
-0.00189 benchmark

and on leg PVs because notionals differ (1 vs 1mln)

-0.0072079492472434375 strata
-7208 benchmark

The difference in fixing rate derives from the data sources. Strata deduces EURIBOR-6M from 6M deposit (I think), while benchmark curve reads EURIBOR-6M @ 2016-7-7 from historical fixings.

It could be fixable via FIX curve node, but “Curve node dates clash”. Is there any other way?

As a side note, I’m not sure that the fixing rate difference is enough to explain an error of 1e-5 (from 1e-13 of the last future).

EDIT: another big difference I found is in the floating leg flows. Strata computes the floating coupon every 6M. The benchmark curve computes only the first flow at start date (using EURIBOR-6M at fixing date). The leg PV is still correct because the benchmark floating leg also include in the PV a fictional discounted payment of the notional at start date and a fictional discounted receivement of the notional at maturity.

I’m not sure we understand how the benchmark is pricing. A payment at the start and end of a swap isn’t enough to generate the PV. The whole point of a swap is the payments every 6 months!

Just to note that the IborFixingDeposit pricer uses the curve to query the value, and never uses fixings. See DiscountingIborFixingDepositProductPricer.forwardRate().

Hi again!
In the meantime they explained me that the wrong cash flows of the swaps in the benchmark curve are a side effect of the application. Long story short, it is possible to see the right 6M cash flows used for pricing the swap, but only after a full repricing routine.

After that I compared the 6M payments of Strata EXPLAIN_PRESENT_VALUE with the benchmark ones. The fixed leg is the same, but the floating leg is not. It is quite obvious, since the discount curves used for pricing are different.

Also payment dates and fixing dates are the same, but floating rates (and discount rates) are not.
I also tried forcing the first fixing with a single point time series when building market data

.addTimeSeries(IndexQuoteId.of(IborIndices.EUR_EURIBOR_6M), tsEur6)

and the Strata curve is now exact until the 4Y swap (error 1E-13). EXPLAIN_PRESENT_VALUE also displays the correct first fixing. But from the 5Y swap the error jumps immediatly to 1E-5.

Eventually, I found something interesting. These are the fixed flows for the 5Y swap in benchmark and in Strata

Payment date,Fixed rate,Disc factor,Disc Flow,Accr Days, ,Payment date,Fixed rate,Disc factor,Disc Flow,Accr Days
bench,bench,bench,bench,bench, ,Strata,Strata,Strata,Strata,Strata
, ,
2017-07-11,-0.165,1.0022355066705,1650.00000000004,360, ,2017-07-11,-0.165,1.00223550666938,1653.68858600447,365
2018-07-11,-0.165,1.00639377773059,1650.00000000004,360, ,2018-07-11,-0.165,1.00639377772873,1660.5497332524,365
2019-07-11,-0.165,1.00720290716756,1650.00000000004,360, ,2019-07-11,-0.165,1.00724906172884,1661.96095185259,365
2020-07-13,-0.165,1.00865977383596,1659.16666666666,362, ,2020-07-13,-0.165,1.00870601729594,1673.61140036353,368
2021-07-12,-0.165,1.00830380590786,1645.41666666662,359, ,2021-07-12,-0.165,1.00835146004766,1659.15829822009,364

As you can see, the accrual days are different. Strata seems to use ACT/356 on the fixed leg. This makes no sense, because the corresponding ResolvedSwap object has the right convention on the fixed leg:

dayCount=30U/360
// extracted from eclipse debugger

Is that a bug in Strata?

EDIT: despite the different day count in EXPLAIN_PRESENT_VALUE, the yearfraction is perfect.

Yes, I’ve checked and DiscountingRatePaymentPeriodPricer line 448 uses a simple count of days rather than the DayCount.days() method. (because DayCount.days() was added later). This shouldn’t have affected the PV because the year fraction is correct.

BTW, is it possible that the benchmark system does not use a multi-curve framework? That might cause the discount curve and forward curve to drift apart over time, which might explain the difference (assuming that your Strata curves are making use of the multi-curve framework).

Yes, the yearfraction calculations are perfect so the wrong daycount is only a “printf” problem.

Since the standard discount curve must be self consistent, I put in groups.csv only 1 group and 1 curve. Am I using the multi-curve framework anyway?

EUR-Standard,Discount,EUR,EUR-STD
EUR-Standard,Forward,EUR-EURIBOR-3M,EUR-STD
EUR-Standard,Forward,EUR-EURIBOR-6M,EUR-STD

I also tried the same curve calibration (same valuation date, same input market data, same interpolation settings and so on) with Murex and the result is equal to the benchmark. Strata is still different.

The strange thing, again, is that everything goes well until the 5Y swap. So the error on 3Y swap and 4Y swap is below 1E-12 (with correct euribor fixing via timeseries). On 5Y swap it jumps immediatly to 1E-5. Here is an extract of the comparison table.

Tenor,Benchmark,Strata,Abs_Error
MAR 18,-0.314339484782413,-0.314339484687837,9.45760691983821E-11
JUN 18,-0.326147187951814,-0.326147187870456,8.13580314229512E-11
3Y,-0.238365595744974,-0.238365595745062,8.7957419125928E-14
4Y,-0.214533740288634,-0.214533740288516,1.18016707517654E-13
5Y,-0.164848412392087,-0.164875587775754,2.71753836670052E-05
6Y,-0.0920400139575374,-0.0920795485117632,0.000039534554225798

As a side note, I also tried using the benchmark curve in Strata for picing the 5Y swap and the PV (global and legs’) is correct, with an error of 1E-10. So the the problem is not in the pricing method.

We’d really like to use Strata for calibration, since it is extremely fast and lightweight, but we need a precision of 1E-4 bps at least. Unfortunately, for now, the error on swaps is worse. And it is difficult to track down why, since the settings on trades and curves appear to be the same.


P.S.: I found now via debugger that when calling

marketDataFactory().create()

for creating a calibrated market data, the timeseries I added are present, but when calling

calibrate()

the timeseries are gone. This causes the error jump to happen on the 3Y swap. When I manually call calibrate() with the correct marketdata (timeseries included), the error jump happens on the 5Y swap as I said.

Your groups.csv file shows a single curve shared between discounting and forward. So, although you are using the multi-curve framework, it is calibrating just one curve for all three purposes. As such, my guess that the discounting and forward curve might separate is wrong.

I’m not sure where to take this next. Perhaps the next step is to calibrate a simpler curve on both systems? One that only has swaps and no futures or short tenor instruments. That way, it might confirm that calibration of swaps alone is compatible.

If you want to paste your code where you saw the time-series disappear, I can take a look at that.

For now, the aim is to calibrate a single curve (the standard eur discount) and I’m pretty sure that both system are using a single-curve framework for this task.

The disappearing of the timeseries from marketData occurs during the MarketDataFactory routines. When using a “csv approach” for calibration, I use

MarketData calibratedMarketData = marketDataFactory().create(reqs, marketDataConfig, marketData, refData);

for building the calibrated marketData (and eventually print the curve parameters). The marketData input here have the fixing timeseries included. When the process calls for the calibrator, the first instruction is

Map<Index, LocalDateDoubleTimeSeries> timeSeries = marketData.getTimeSeriesIds().stream()

and here marketData has no timeseries at all.

I then writed a more “direct” approach:

ratesProvider = calibrator.calibrate(curveGroups.get(0), marketData, refData);

where marketData has the timeseries included. The resulting curves are different on 3Y and 4Y swaps.

So, somewhere inside the MarketDataFactory the timeseries are filtered out. Maybe it is caused by reqs? The measures to be calculated are

List columns = ImmutableList.of(
Column.of(Measures.PRESENT_VALUE),
Column.of(Measures.EXPLAIN_PRESENT_VALUE),
Column.of(Measures.LEG_PRESENT_VALUE));

Time-series bug:
https://github.com/OpenGamma/Strata/pull/1266

I know it would never happen in practice, but at this point I want to try this calibration supplying the 6M forward curve from an external source.
Is this possible? I can not find a way to call the calibrate() method in the right way.

The basis idea is to build a group of curves in which the discount curve carries also the 3M forward rate but not the 6M forward rate:

EUR-Standard,Discount,EUR,EUR-STD
EUR-Standard,Forward,EUR-EURIBOR-3M,EUR-STD
EUR-Standard,Forward,EUR-EURIBOR-6M,external_curve

and the external curve is just a complete and static InterpolatedNodalCurve class.
But since the calibrate() method needs a CurveGroupDefinition, it seems impossible to pass the external_curve, because it has not a corresponding CurveGroupDefinition (because it doesn’t need to be calibrated).

It should be possible to calibrate with a fixed 6 month curve, however, the method you need to call is package scoped.

  ImmutableRatesProvider calibrate(
      List<CurveGroupDefinition> allGroupsDefn,
      ImmutableRatesProvider knownData,
      MarketData marketData,
      ReferenceData refData) {

If you call this by reflection, or by altering the Strata source, you can pass in an ImmutableRatesProvider of “knownData” that contains the 6 month curve and it should calibrate the discount and 3 months against it (untested, but looks like it should work).

So, the groups.csv file for calibration would then just be:

EUR-Standard,Discount,EUR,EUR-STD
EUR-Standard,Forward,EUR-EURIBOR-3M,EUR-STD

Thank you. I think it worked because now Strata does not complain about missing EUR-EURIBOR-6M, but the newton-like method does not converge anymore.

EDIT: also tried with both 3M and 6M forward curves supplied via RatesProvider, but the method does not converge anyway.

Still not a logical cause of this error behaviour found.

I’ve spoken to our head quant, and the only thing we can think of that might result in a sudden jump would be a mismatch in holiday data. Whether that is the case I don’t know, but it might be worth investigating. (We don’t have anything that hard codes 4 or 5 years and behaves differently).

The converging problem suggests that there is a node in the 3 month curve that depends on 6 month LIBOR (which you had changed to be fixed, so could not be varies to find convergence).

Thank you very much for your effort.
After further investigation, I found that the different results are caused by the benchmark curve calibration.
During the calibration, the swap products are actually valuated without recalculating all the payments of the floating leg. So there is a notional payment at start and a discounted negative notional payment at the end. Forcing the benchmark software to recalculate all the payments leads to a more Strata-like curve:

The resemblance is impressive now, in the order of 1E-8 bps.

Not so lucky with OIS swaps, though. After many efforts, I found that Strata conventions on OIS swaps (EONIA) have 1 day of delay for payments. So, I created a new convention for fixed term OIS and 1Y OIS:

public static final FixedOvernightSwapConvention EUR_FIXED_TERM_EONIA_OIS_0DAY =
makeConvention(“EUR-FIXED-TERM-EONIA-OIS-0DAY”, EUR_EONIA, ACT_360, TERM, 0, 2);

Now the discrepance is very low until 30Y (1E-7 bps, very good!). After 30Y there is something wrong… Maybe it could be caused by the longer time gap between points?

Looks good!

Perhaps interpolation is only really used after 30Y because you have so many other shorter-tenor nodes? Maybe the interpolator differs? Or perhaps the holiday data differs that far out?