Export AI Factor validation

I'm trying to export a AIFactorValidation with Factor list export, but I keep getting the message "No predictions are available on 05/27/2026" even though I have not included that date in the factor export (using the dates that has a factor value only). Get the same message if I pick just one date to export.

Hi,

Any reason why you are going through Factor List to download validation prediction data? Did you try downloading the validation data from the model itself? It's faster and there's no API costs.

It's kind of hidden right now. You have to click the three dots, then select "Download Predictions". In my example below I did not save the data so it's grayed out (we're going to make this download option a lot more obvious soon)

As far as the error... Factor List may be doing a dummy call using current date to figure out how big the universe is which causes the error. It should use the end date of the period.

Thanks, it works. First time I tried it I thought it did not work, but it just took a very long time.

The reason I want the prediction is that I want to use them as pre-training. If I use the AIFactorValidation formula in the Universe to filter out stocks, it can not be used in the next AI factor training, for some reason I get the message "Could not determine lookahead" when I try to load data. I probably should have started to raise that as problem.

Converting the AIFactorValidation to a Imported Factor then use it in the Universe seems to work. A bit of a workaround...

The reason is that there’s a conversion going to to csv format. It’s the conversion to csv that can take several minutes, and really makes no sense to do. We should just give you the parquet file immediately, then you can decide if you need it in csv.

This is the approach we are considering for most downloads that will involve larger datasets.

Let us know what you think about parquet format downloads.

Also, we’ll investigate why it’s not possible to use AIFactorValidation in the universe. Might be an easy fix.

Thanks

When I'm on a Starlink connection I have problems downloading parquet, csv is fine can be used with a download manager, but parquet just keeps failing and can't be resumed. It look like you are using a Nginx server, I know from experience that they are tricky to set up for large files. It look like Accept-Ranges or Content-Length is not set properly or something else that does not allow to resume a download, or maybe you have a too narrow timeout setting.
You mentioned that the csv files is generate through a file conversion, is the parquet files generated on the fly? Maybe that is why the file can't be resumed.

@AlgoMan I had problems downloading parquet files as well. I solved it linking the connection with a LAN wire, download speed increase and in minutes I got the archive. I hope it helps

It’s a very long wire to lower orbit :woozy_face:

We’re moving away from dynamically generating parquet files, or any other big file. They will be actual files in our network storage. This should make it much more reliable and compatible with download managers.

But of course, parquet files are in binary format. So to read it you will need to convert it. We are exploring different options to make downloads better.

1 Like