NEW: Data Miner App & P123 API -- v1.0 (beta)

Don’t want to speak for everyone else but for me, I’d like to pull 20 factors for 3000 stocks on a weekly basis for 5 to 10 years. That would be the same amount of data used in a single simulation run for me anyways.

If I understand correctly, 3000 stocks for one week for one factor ~ one API request. So to do what I want I need 1 API request * 20 factors * 52 weeks * 10 years = 10,000 API requests…

This doesn’t work I get this error. Can you please provide the working code plz:

2020-12-24 15:13:07,294: Output is empty
2020-12-24 15:13:07,289: Iteration 1/1 run 1/2: failed
2020-12-24 15:13:07,288: API request failed: ERROR: Ranking failed on 01/01/2000: Cross sectional formula functions only be used relative to ‘Universe’ in ranking systems

Main:
Operation: RankPerformance
On Error: Stop

Default Settings:
Vendor: Factset
PIT Method: Prelim
Buckets: 1
Start Date: 2000-01-01
End Date: 2020-12-21
Rebalance Frequency: 4Weeks
Universe: International Equity Strategy
Benchmark: VEA
Minimum Price: 1

Iterations:
-
Name: EarnYield_I #Industry 1
Ranking:
Nodes:
-
Type: StockFormula
Name: a
Formula: “Aggregate("EarnYield",#Industry,#Avg,16.5,#Exclude,False,True)”
Rank: Higher
Scope: Industry

Hello,

I wanted to know if you would allow us to use FHist with negative values to get technical data with the API for users without data license?
Currently, there is no way to get future volume for example. Want to try it as well as other data as labels in ML models.

FHist support both positive or negative values. The documentation was updated (but perhaps not clear enough?)

However Volume factors & functions need not be restricted. It’s technical data and it does not come from Factset. So it would be similar to what we did with FutureRel%Chg & Future%Chg. You can get the exact future values.

Can you give an example of what future volume metric you need?

I would like to buy more API requests, but from the other API thread I see that P123 is planning to go from a subscription model to a “a la carte” model, when will this happen? Should I just upgrade my API subscription now, or should I wait for the new system? (Also I get an error message telling me that I’ve reached my API limit when trying to download data, even though I’m supposed to have 1k request remaining for the month).

Apparently tomorrow Marco told me today. I want to buy the 500K requests package but it’s not available yet. Let’s check back tomorrow if the option appears.

Marco,

Using the “additionalData” option with this function: (any function within FHist)
FHist(“AvgVol(20)”, -4)

returns “A data license is required for this operation”

My understading is that it should not

Looks like it’s in place now - I think I’ll go for the 500K option right away :slight_smile:

Well, I bought 500k extra. The dashboard did not update - my limit is still stuck at 10k, but I am able to use the API again so I guess this is only a display bug.

I also notice that the API “price” for downloading ranks is now much lower - thanks!

Dashboard only reflects usage over monthly quota for now, will be changed to include add-on extra request credits as well.
However, you can see detailed API quota info (including purchased add-on statistics) in the DataMiner & API tab under account settings.

Marco, you said that volume factors & functions need not be restricted. It’s technical data and it does not come from Factset.

But when using for example, FHist(“AvgVol(20)”, -4), in the “additionalData” option field, with the “ranks” API, I am getting this error:
A data license is required for this operation

Please fix this. Thank you.

we don’t allow FHist , that’s the problem. We could but then we need to look inside the quotes to make sure it’s not factset data.

Much easier (and clearer to use) for us to supply future functions/factors for volume. Like we did with Future%Chg and FutureRel%Chg.

We could start off with FutureAvgVol(bars [,offset]) and FutureVol(bars)

So for example

  1. to get the volume 10 trading days in the future of the as-of date you would use: FutureVol(10)

  2. To get the average volume for the future 10 trading days you use: FutureAvgVol(10)

  3. To get the average volume for the future 10 trading days skipping the next month (21 bars) you use: FutureAvgVol(10,21)

Will this work? we can add relatively easy.

Thanks

Yes, it will work but being able to get data with negative FHist would be much much better since this allow us to create much more labels and we would not need to have a new future function every time we to access a technical data.

1/ If negative FHist is not possible then, it would be great to have a function to get future volatility, HighVal, HighValBar, LowVal, LowValBar and correlation. Maybe allow negative values within these existing functions without adding new Future functions.

2/ The “Loop” functions such as “LoopMax” are capable of looking inside the formula and detect whether it is a factset data or not. Why “FHist” would not?

I have tested with LoopMax using technical data and it works. When using a fundamental data then it returns “A data license is required for this operation”.

Thank you.

OK . we’ll see how feasible it is to allow FHist.

Warning to everyone using the Python P123 API. I purchased extra API requests, and during my data pull, the P123 API threw a “not enough API requests left” error for no reason. I wasn’t even close to the limit. The result was that all the data I had pulled up to that point was lost but I had still “spent” the API requests.

Hi Phil,

The only error we see on our end is “Connection reset by peer”. This usually means there was a lost connection between our server and your application. Our API has automatic retries but it cannot recover if the connection is shut down by the peer. Do you have any logs that show the “not enough API requests” so we can dig deeper? Also, your application should save after every iteration, and be able to resume from a failure, to avoid losing work.

So at this point we can’t see issues in our end but we’re happy to apply the 50K api credit you requested.

Thanks for your feedback.

NVM phil . we do see a potential issue that caused “not enough API requests” . Sorry about that

Marco - I don’t know what philjoe is actually trying to do, but he is an early adopter. If he has troubles then it is likely that others will as well. And if you want this to succeed then you need to be sympathetic to the needs of early adopters. So make sure you are not throwing your apps over a brick wall and telling customers to deal with it. For Big Data projects it seems reasonable that the driver “P123 API” (whatever) should take care of saving intermediate work. Or there should be a library of support software to deal with such things. Maybe if philjoe opens up about what he is attempting to accomplish then the community can chip in to help.

Steve

Steve I was just using your python script to do a big data pull. It was interrupted by an error, when it should not have been. Because of the interruption all of the pull was lost.

Marco, you’re right, in an ideal world each iteration of the data pull would be saved to hard disk. But because I don’t have top of the line equipment that write-process would take way too long.