Filecoin v2 retrievals

Here Be Dragons!

Filecoin v2 retrieval is still in progress! If you experience issues with this format, please open a Github ticket with as much information as you can so we can debug faster!

A new piece-based splitting format simplifies Filecoin retrievals.

This allows for a much more convenient configuration of Filecoin retrievals and supports other Filecoin protocols like bit swap and graphsync.

To configure Filecoin v2 retrievals, you specify a Subset CID (a Subset is a range of Slots in an Epoch). For the OF1 archive, we intend to publish Subset CIDs in an easy-to-retrieve place. You can also specify an optional list of storage_providers from which you want to fetch the data.

version: 1
epoch: 100
data:
  filecoin:
    enable: true
    subset_cids: 
      - bafyreibqt2nvroysxlxctgb52xxn27ectsllv2xyka4qar7ga6vupmbs3i
      - bafyreibqt212roysxlxctgb52xxn27ectsll2xsfk24qar7ga6vusdjn32
    storage_providers:
      - f02366527
indexes:
  slot_to_cid:
    uri: /of1/100/epoch-100-bafyreibqt2nvroysxlxctgb52xxn27ectsllv2xyka4qar7ga6vupmbs3i-mainnet-slot-to-cid.index
  sig_to_cid:
    uri:  /of1/100/epoch-100-bafyreibqt2nvroysxlxctgb52xxn27ectsllv2xyka4qar7ga6vupmbs3i-mainnet-sig-to-cid.index
  sig_exists:
    uri: /of1/100/epoch-100-bafyreibqt2nvroysxlxctgb52xxn27ectsllv2xyka4qar7ga6vupmbs3i-mainnet-sig-exists.index
  gsfa:
    uri: /of1/100/epoch-100-bafyreibqt2nvroysxlxctgb52xxn27ectsllv2xyka4qar7ga6vupmbs3i-mainnet-gsfa.indexdir