TL;DR: The authors of this paper using Bootstrapping to get p-values along with the Benjamini-Hochberg Procedure (BH) to get a false discovery rate (FDR) correctly state that they get a different result from a paper using Bayesian analysis: Is There a Replication Crisis in Finance?
Both papers are ultimately about False Discovery Rates (FDR) so there is a true disparity in the results of the papers, I believe.
The latter paper has been sited frequently in the forum. I first became aware of it when Yvual mentioned it and he has mentioned it since as well as blogging about it I think:. E.g, in this thread: Why such a large discrepancy between backtesting and real-life results?. it is an interesting paper and I have also discussed it in the forum. I don’t mean to imply that Yuval or I endorse or use anything within the paper or what we might use if we do use some of it. Probably we find some similar lessons as well as some different lessons from the paper.
So which paper is right? Lots of good factors or surprisingly few?
One could probably explore this on their own with P123 data. The BH is not that hard to do. The bootstrapped p-value could be a little challenging. I am not sure that there is a Python library that does this (although there is a library in Scikit-Learn that gives confidence intervals using bootstrapping), but you can find multiple examples of code to do this with a google search. I believe R does this and a bootstrapped p-value could be obtained using an iterative approach over at JASP.
Here is a simple video by Stat Quest explaining getting a p-value using Bootstrapping: Using Bootstrapping to Calculate p-values!!! For someone on the forum who has sent me links about Stat Quest (you know who you are or if you have ever watched the Stat Quest videos) I cannot help but say: Bam!
Some Bayesian analysis can be found at JASP, PyMC3 with Python or over at R using the BEST program.
Jim