Can AI Really Help with Quant Investing? I’m Putting It to the Test

Evidence and scope — Roles, risks, and planned experiments

External research informs the discussion of AI roles in prediction, research, and automation, along with their risks. Comparative experiments are planned work. Time savings in external studies of document work are not this project’s results or evidence of investment returns.

From the limited predictive capabilities of machine learning to LLM-assisted research and automation, this article examines what individual investors can realistically expect—and which risks they must check.

When you spend enough time reading regulatory filings, cleaning price data, and writing backtest code, one question naturally comes to mind:

“Could AI help me make better investment decisions and conduct better quantitative research?”

If research and calculations become faster, it seems reasonable to expect that we could test more hypotheses and discover patterns people might otherwise miss.

But this is where we need to pause. Does faster also mean more accurate?

A field experiment found that generative AI can reduce the time required for some document-related knowledge work. But that study did not measure investment research quality or investment returns.[S9] Producing a report draft more quickly is entirely different from improving investment performance after costs.

Rather than accepting someone else’s success story or an impressive-looking backtest at face value, I decided to run my own comparisons. Throughout this series, I will examine exactly what AI improves, one category at a time.

Does AI reduce research time, lower the number of errors, reduce prediction error, or actually improve risk-adjusted performance after costs?

This article is the starting point for that experiment. It divides AI’s potential roles in quant investing into prediction, research, and automation, then outlines the risks we need to understand before trusting the results and the comparisons I plan to run.

This series will not recommend individual securities or provide buy-and-sell timing instructions. It will also not address personalized asset allocation, tax or legal issues, or guarantee returns.

Quant Investing Is Not a Magic Way to Predict the Future

When people first encounter quant investing, they often picture complicated equations or automated trading systems. But the kind of quant investing discussed here is a much simpler process:

1. Define a hypothesis to investigate.
2. Prepare the data needed to test it.
3. Specify trading and risk-management rules.
4. Test the hypothesis using only information that would have been available at the time.
5. Evaluate the results, including costs and the possibility of failure.

In other words, quant investing is not a way to forecast the future perfectly with numbers. It is an approach that connects hypotheses, data, rules, and validation. AI does not replace this entire process. It is a tool that may be useful at certain stages.

The outputs of machine-learning models are generally limited as well. Based on findings from asset-pricing research, examples include the following:[S1]

• An estimate of expected return or a risk premium under specified conditions
• A relative expected-return score across multiple assets

In asset-pricing research, machine learning has been used to capture nonlinear relationships and interactions among variables. In some studies, certain tree-based and neural-network models performed better than linear models in out-of-sample evaluations.[S1] But those findings came from particular U.S. market datasets and specific research designs. They do not mean that an individual investor will achieve the same performance by using the same model, nor do they mean that AI can guarantee profits.[S1]

More importantly, a prediction model and an investment rule are not the same thing.

Suppose a model estimates that an asset has a 55% probability of rising. That number alone does not tell you whether to buy it. You still need to determine whether the probability is well calibrated, how large the potential loss may be, how much capital to allocate, how frequently to rebalance, and whether any advantage remains after transaction costs. This practical distinction follows from the principle that a model’s purpose and limitations must be stated clearly while costs and risk are managed separately.[S4][S6]

AI is not a complete decision-making system. At most, it may provide one score or estimate that feeds into such a system.

Not All “AI” Is the Same: Machine Learning, LLMs, and Conventional Automation

If we group every technology under the single label “AI,” it becomes difficult to determine what worked and where errors occurred. In quantitative research, we need to distinguish at least three categories.

Traditional machine learning can primarily be used to estimate conditional expected returns or risk premiums from historical numerical data and calculate relative scores for assets.[S1]

Generative AI and large language models, or LLMs, need to be evaluated separately. Some general knowledge-work studies have found that they can reduce the time required for certain document tasks, but that does not prove that they improve the quality of investment research or investment returns.[S9] They may also be useful in financial text processing, but apparent accuracy can be inflated by data imbalance, while finance-specific models may struggle with instruction following and language generation.[S10] LLMs can also generate plausible-sounding claims that are not true.[S7]

Tasks such as downloading data and checking for changes can be automated with conventional software even without AI.[S8] For example, the U.S. Securities and Exchange Commission’s EDGAR system provides company filing histories and some XBRL financial data through JSON APIs, making it possible to programmatically collect and monitor changes in U.S. filings.[S8]

The reason this distinction matters is simple:

An LLM’s ability to work naturally with documents and its ability to predict future investment returns are separate claims.[S10]

We should not assume that a model that handles documents well can also generate excess returns. Nor will I treat its ability to assist with coding and explanations as an established fact. In later experiments, those capabilities will be tested using measures such as test pass rates, reproducibility, correction time, and error counts.

Role ① Making Limited Predictions About Prices and Risk

The first potential role of machine learning is to turn questions about prices or risk into limited, measurable targets that can be estimated.

This series will begin by distinguishing among targets such as:

• Classifying whether the next period’s return will be above zero
• Estimating an expected return or risk premium under specified conditions
• Ranking multiple assets by relative expected-return scores
• Investigating whether future volatility or the probability of a large loss can be defined as separate experimental targets
• Investigating whether classifying specific market regimes could be a useful experimental target

Of these possibilities, asset-pricing research directly supports the more limited use cases of estimating conditional expected returns and risk premiums and producing relative scores.[S1] This article does not present volatility, large-loss probability, or market-regime prediction as capabilities that have already been proven effective. We first need to determine whether they can be defined as appropriate targets for later experiments.

Even in these applications, a model does not know the future. It simply compresses relationships observed between conditions and outcomes in historical data, then assigns a probability or score to new inputs.[S1]

Tree-based models can capture nonlinear relationships and interactions among variables.[S1] Asset-pricing studies have also shown that tree-based models and neural networks may be able to make use of these relationships.[S1]

But the ability to represent complex relationships does not guarantee that those relationships will persist. The more sophisticated a model appears, the greater the temptation to try more configurations. Repeatedly testing different settings increases the risk of selecting a backtest winner that merely fits chance features in historical data.[S3]

Predictions must also remain separate from trading rules. Turning a probability or score into an actual trade requires an additional set of decision rules. The specific conditions for doing so will be fixed together in the comparison design described later.

There is also no basis for declaring in advance whether logistic regression or a tree-based model will perform better. Results may depend on the data, prediction target, period, model settings, and costs. That is exactly why they need to be compared under the same conditions.

Role ② Assisting with Investment Research

An LLM should not be treated as an automatic stock picker, but it may be worth evaluating as an assistant in the research process. A field experiment showing that generative AI reduced the time required for some general knowledge-work document tasks indirectly suggests that it might save time on research drafts and summaries. It does not prove that LLMs improve investment-research quality, decision accuracy, or investment returns.[S9] Potential benefits have also been observed in financial text processing, but certain benchmarks show that headline accuracy may be inflated by class imbalance and that finance-specific models may struggle with instruction following and language generation. Document-processing ability and investment-performance improvement therefore need to be evaluated separately.[S10]

For example, later experiments could ask an LLM to assist with tasks such as:

• Drafting a list of business segments and risk factors from a lengthy filing
• Suggesting a table structure for comparing common fields across documents
• Explaining unfamiliar financial or statistical terms in plain language
• Generating both a research hypothesis and questions that could disprove it
• Explaining the flow of backtest code
• Suggesting necessary test cases or edge cases
• Recording experimental conditions and revision histories in a consistent format

These are proposed experimental tasks, not confirmed benefits. Later tests will measure whether assistance with code explanations, code generation, and research records saves time, broadens the range of research questions, or reduces errors.

An LLM’s output should therefore be treated as a draft that requires review, not as a finished analysis.

A practical review process should include the following:

1. Open every original source linked in the response.
2. Check figures and units against the tables in the original source.
3. Verify quotations, authors, and publication details.
4. Reproduce calculations independently in code or a spreadsheet.
5. Run generated code on a small sample and through unit tests before using it more broadly.
6. Look for contrary evidence or exceptions omitted from the summary.
7. Base investment conclusions on verified evidence and predefined rules, not on a chatbot response.

LLMs can generate plausible but incorrect facts or sources, and these hallucinations have not been completely eliminated.[S7] Investor alerts likewise warn that AI-generated information may be inaccurate, outdated, or entirely fabricated, and advise investors to check original sources and consult multiple references.[S11]

Based on the evidence currently available, we cannot say that using an LLM improves an individual investor’s long-term returns. This review also did not find an established average error rate for quant code written by generative AI. The value of coding assistance should therefore be measured using test pass rates, reproducibility, human correction time, and error counts—not vague user satisfaction.

Role ③ Automating Repetitive Data, Code, and Reporting Tasks

Official APIs and conventional software can automate repetitive work such as data collection and change monitoring.[S8] Generative AI may also reduce the time required for some general document tasks, but that is not comparative evidence that automation creates more value than prediction in investment research.[S9]

Many tasks are reasonable candidates for automation:

• Collecting filings and data from official APIs on a defined schedule
• Updating price and financial data
• Checking for missing values, duplicates, and anomalies
• Running backtests for multiple models under identical conditions
• Producing results tables before and after costs
• Recording model versions and changes in assumptions
• Saving experimental results and error logs in a consistent format

SEC EDGAR provides APIs and bulk files that make company filing histories and some XBRL financial data available without an authentication key.[S8] The collection of U.S. filings and monitoring of new submissions can therefore be turned into a repeatable software process.

But an official API does not necessarily provide data that is immediately ready for analysis. Users still need to check differences in company-specific XBRL tags, amended filings, and reporting periods.[S8] Even if an LLM writes the collection code, the user remains responsible for validating the meaning and timing of each field.

The objective of automation should not be limited to reducing the number of clicks. We also need to verify that the same inputs produce reproducible results, that errors can be traced to the stage where they occurred, that changes to data, models, and cost assumptions are versioned, and that the items requiring human review are clearly identified.

Five Risks to Understand Before Using AI

This article introduces the basic concepts behind five key risks. Specific methods for preventing leakage, conducting walk-forward validation, controlling repeated experimentation, and calculating costs will be covered in later installments.

1. Data Leakage: Mixing the Test Answers into the Training Material

Data leakage occurs when future information that should not be available during testing enters the training process. It is like letting a student see the answer key before asking them to take the same exam.

Applying a conventional random split to time-ordered data can result in a model being trained on future observations and then evaluated on an earlier period.[S2] With financial time series, the model should be trained on earlier periods and evaluated on later ones. The final judgment should be based on a period that was not used for model selection.[S2][S6]

A time-series split does not automatically solve reporting delays, retrospectively revised data, or survivorship bias.[S2] Even after sorting the date column, we still need to check when financial information actually became public, whether the data was later revised, and whether the dataset includes only securities that survived. [S2] explains why chronological splitting is necessary, but it does not provide solutions to all of these finance-specific data problems.

LLMs introduce an additional blind spot. With closed models, outside users may be unable to determine fully whether the training data included information published after the historical evaluation date. An experiment that gives a current LLM old news or filings and asks it to predict what would happen next therefore cannot completely rule out temporal leakage within the model itself.

2. Backtest Overfitting: Choosing Only the Best-Looking Result

Imagine changing periods, variables, trading thresholds, and model settings hundreds of times, then selecting the best result. The selected strategy might reflect a genuine pattern, but it might also be the one result that happened to fit historical noise among a large number of attempts.

As the number of tested configurations increases, so does the risk of selecting an unusually strong backtest by chance.[S3] This is not limited to complex models. Even a simple moving-average strategy can be overfit if its periods and thresholds are repeatedly changed after reviewing the results.[S3]

To reduce this risk, we need to record the original hypothesis, the range of variables and settings tested, the total number of attempts, any changes made after seeing results, and a final test period that was not used for model selection. If we keep only one successful result and delete failed experiments, we lose the ability to see how many attempts were required to find that apparent success.

3. Market Change: Yesterday’s Map Is Not Today’s Terrain

A model is like a map that compresses historical data. But market conditions and participant behavior can change. We cannot assume that a relationship observed in the past will continue with the same strength in the future.

Model-risk management guidance for financial institutions emphasizes out-of-time and out-of-sample evaluation, data-quality checks, ongoing performance monitoring, and recalibration or redevelopment when necessary.[S6] This guidance is not a rule that directly applies to individual investors, but adapting its validation principles to personal quant research is a useful practical inference.

This review did not identify a single best algorithm for detecting market change in every dataset. Rather than assuming that adding one regime-change detector solves the problem, we need to keep observing how performance and prediction error vary over time.

4. Transaction Costs: The Difference Between Backtested and Achievable Returns

For the return shown in a backtest to equal the return in a real account, trades would need to execute at the assumed price without cost. That is not how real markets work.

Commissions, the bid-ask spread, and market impact—the effect of an order on the price—are costs arising from execution. Turnover is not itself a cost; it measures how frequently holdings are replaced and affects trading frequency and the total scale of costs.[S4]

A study of actual institutional orders found that trading costs and strategy capacity differed across strategies and that execution designs intended to reduce costs could change net returns.[S4] Because those findings were based on historical institutional order data, they cannot be applied directly to individual investors or the Korean market.[S4]

The point is not that every strategy fails once transaction costs are included. It is that we cannot judge whether a strategy is executable until costs are included.

This review did not find a comparison of AI strategies that incorporates all commissions, taxes, spreads, and market impact faced by Korean individual investors. Later experiments will need to define the target market and order size, then fix an appropriate set of cost assumptions in advance.

5. LLM Hallucinations: Delivering Wrong Answers with Confidence

An LLM is not a database that guarantees the factual accuracy of every sentence. It may confidently provide nonexistent papers, incorrect figures, or fabricated citations.[S7]

In investment research, users should verify that recommended papers exist and confirm their bibliographic information from the original publication. Numbers in filing summaries should be checked against the original tables. Calculations should be reproduced independently, generated code should be tested using small samples and unit tests, and users should separately search for contrary evidence omitted from the answer.

U.S. securities regulators identify phrases such as “an AI trading system that can’t lose” or “guaranteed stock winners” as warning signs of investment fraud.[S11] The SEC has also brought an enforcement action in an individual case involving allegedly false and misleading claims about AI-powered automated trading and performance.[S12] That case does not discredit every AI investment service, but it does show that the label “AI” is not evidence of trustworthiness.

The Four Approaches I Plan to Compare

To evaluate the value of an AI model, we should not make AI models compete only against one another. They need to be compared under the same conditions with baselines that do not rely on complex technology.

1. Buy and Hold

The first baseline is to buy a defined asset and continue holding it.

This is not a “do nothing” strategy. It provides a minimum benchmark for comparing complex strategies with the result available without frequent trading or ongoing model maintenance. The specific investment universe and holding rules will be fixed before the experiment begins.

2. Simple Momentum

The second baseline is an explicit non-AI rule based only on past returns.

The daily momentum factor in the Kenneth French Data Library subtracts the return of a loser portfolio from that of a winner portfolio, with the portfolios formed using prior returns from months 2 through 12.[S5] This is an academic example of how momentum can be implemented; it does not automatically determine the rules for the later experiment.

For the actual experiment, I will predefine the target market, whether the strategy is long-only, the return-measurement period, the security-selection criteria, and the rebalancing schedule. I will also avoid generalizing results from a U.S. academic long-short factor to individual investors or other markets.[S5]

3. Logistic Regression

The third approach is logistic regression, a simple machine-learning baseline that targets whether the next period’s return will be positive.

I will not assume in advance that it will perform poorly simply because it is simple. To determine whether a complex model adds real value, the models must be compared using the same data and evaluation conditions.

4. Tree-Based Models

The fourth approach is a tree-based model capable of capturing nonlinear relationships and interactions among variables.[S1]

Tree models have shown promise in certain asset-pricing studies, but I will not assume those findings will be reproduced using an individual investor’s data.[S1] The questions are whether they improve on logistic regression and, if so, whether the improvement is large enough to justify the additional cost of validation and maintenance.

Conditions to Fix in Advance for a Fair Comparison

To compare the four methods fairly, we cannot give each model a testing environment designed in its favor. As many of the following conditions as possible should be fixed in advance:

• The investment universe and criteria for including or excluding securities
• The point in time when each data item actually became available
• Training, validation, and final test periods
• The prediction target and holding period
• Rebalancing frequency
• Entry, exit, and position-sizing rules
• Methods for handling missing values and outliers
• The search range and number of trials for each model’s settings
• Assumptions for commissions, spreads, and market impact
• Criteria for accepting or rejecting results

With time-ordered data, models should be trained on past periods and evaluated on later ones. Final evaluation should be performed on an untouched period that was not used for model selection.[S2][S6] The same cost assumptions must be applied to every method.[S4]

The evaluation will not stop at a single return figure. I will record returns before and after costs, volatility, maximum drawdown, turnover and trade count, stability across periods, and the size of any improvement over the baseline. AI’s effect on the research process will be measured separately from investment performance using factors such as research and maintenance time, error counts, and the number of manual corrections.

The decision to assess costs, stability across periods, and ongoing performance monitoring together is a design principle derived from research on trade execution and model-risk management.[S4][S6] I will not assume that any one metric can perfectly determine which strategy is best.

Eight Questions That Will Guide the Entire Series

To avoid choosing favorable interpretation criteria after seeing the results, here are the questions I will ask in advance.

1. Do AI models actually outperform buy and hold and simple momentum on unseen data? Machine learning has shown potential, but there is no guarantee that it will beat simple baselines.[S1][S2][S5]

2. Is the observed difference merely a lucky winner selected after repeatedly testing many configurations? The more trials we run, the greater the risk of backtest overfitting.[S3]

3. Was every input genuinely available at the prediction date? In addition to chronological order, we need to check publication dates and whether data was later revised.[S2][S6]

4. Does the improvement remain after commissions, spreads, market impact, and turnover are considered? Execution feasibility cannot be judged before costs are included.[S4]

5. Is performance concentrated in one particular bull or bear market? We also need a way to monitor continued performance deterioration after markets change.[S6]

6. Were LLM-generated summaries, code, and citations checked against original sources and tests? A natural-sounding response is not the same as a factual one.[S7][S11]

7. What exactly did AI improve? Research time, error counts, prediction error, and risk-adjusted performance after costs should be measured separately. Time savings observed in general document work must not be reinterpreted as evidence of improved investment performance.[S8][S9]

8. Does the improvement from a complex model justify the costs of development, validation, explanation, and maintenance? Even if a statistical improvement is observed, the burden of ongoing monitoring and redevelopment must also be considered.[S1][S6]

Preliminary Conclusion: AI Is a Validation Tool, Not a Substitute for Judgment

For now, the most defensible answer is that AI can help under certain conditions.

Machine learning can serve as a limited tool for estimating conditional expected returns or risk premiums from numerical data and calculating relative scores for assets.[S1] Generative AI has shown the potential to save a limited amount of time in general document work, but that finding cannot be interpreted as evidence that it improves investment-research quality or returns.[S9] Its potential and limitations in financial text processing require separate validation,[S10] and important facts, figures, and citations produced by an LLM must be checked against original sources because of hallucination risk.[S7] Assistance with explaining and writing code or maintaining research records remains a hypothesis to measure in later experiments, not a confirmed benefit.

Conventional software and official APIs can automate repetitive procedures such as data collection and change monitoring.[S8] But automation does not guarantee more accurate investment decisions, and people must continue checking the meaning and timing of the data.

Fast, plausible output does not necessarily translate into better investment returns. The current evidence does not support the conclusion that AI can automatically identify winning stocks or improve an individual investor’s long-term returns. Regulators also identify claims about loss-proof AI systems or guaranteed winning stocks as warning signs.[S11]

This series will therefore examine whether AI performs better than simple baselines, whether it uses only information that was available at the prediction date, whether its benefits persist in a future period that was not used for model selection, whether the result is a chance winner selected through repeated testing, and whether any improvement remains after transaction costs. Even if an improvement is found, I will assess whether its size justifies the burden of development, validation, explanation, and maintenance.

This installment has defined the roles AI might play and outlined the main risks. The next step is not choosing an impressive model. It is building a test environment that does not let the model cheat. The next article will focus on chronological data splits, data leakage, walk-forward testing, controls for repeated experimentation, and transaction-cost calculations to explain why validation must come before prediction.

AI’s value will be judged not by how convincing its answers sound, but by whether it still helps after being compared with simple baselines under identical conditions, using chronologically ordered unseen data and accounting for transaction costs.[S2][S3][S4]

Sources

• [S1] Empirical Asset Pricing via Machine Learning | National Bureau of Economic Research; Shihao Gu, Bryan Kelly, Dacheng Xiu | Published 2018-12, revised 2019-09; published in The Review of Financial Studies in 2020 | https://www.nber.org/papers/w25398 ↩
• [S2] TimeSeriesSplit | scikit-learn developers | Current stable documentation; documentation displayed scikit-learn 1.9.0 when searched and accessed | https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.TimeSeriesSplit.html ↩
• [S3] Pseudo-Mathematics and Financial Charlatanism: The Effects of Backtest Overfitting on Out-of-Sample Performance | Notices of the American Mathematical Society; David H. Bailey, Jonathan Borwein, Marcos López de Prado, Qiji Jim Zhu | Written 2014-04-01, last revised 2014-04-14, published 2014-05 | https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2308659 ↩
• [S4] Trading Costs of Asset Pricing Anomalies | Andrea Frazzini, Ronen Israel, Tobias J. Moskowitz; author-publicized manuscript | First draft 2012-10-23, public manuscript 2012-12-05; later published in the Journal of Financial Economics in 2018 | https://pages.stern.nyu.edu/~afrazzin/pdf/Trading%20Cost%20of%20Asset%20Pricing%20Anomalies%20-%20Frazzini%2C%20Israel%20and%20Moskowitz.pdf ↩
• [S5] Detail for Daily Momentum Factor (Mom) | Kenneth R. French Data Library, Dartmouth College | Data range 1926-11-03–2026-06-30; as of access date | https://mba.tuck.dartmouth.edu/pages/faculty/ken.french/Data_Library/det_mom_factor_daily.html ↩
• [S6] Supervisory Letter SR 26-2 on Revised Guidance on Model Risk Management | Board of Governors of the Federal Reserve System | 2026-04-17 | https://www.federalreserve.gov/supervisionreg/srletters/SR2602.pdf ↩
• [S7] Why language models hallucinate | OpenAI; Adam Tauman Kalai, Ofir Nachum, Santosh S. Vempala, Edwin Zhang | 2025-09-05 | https://openai.com/index/why-language-models-hallucinate/ ↩
• [S8] EDGAR Application Programming Interfaces (APIs) | U.S. Securities and Exchange Commission | Published 2024-06-06, last reviewed and updated 2025-04-08 | https://www.sec.gov/search-filings/edgar-application-programming-interfaces ↩
• [S9] Shifting Work Patterns with Generative AI | National Bureau of Economic Research; Eleanor Wiske Dillon, Sonia Jaffe, Nicole Immorlica, Christopher T. Stanton | Manuscript dated 2025-05-06 | https://www.nber.org/system/files/working_papers/w33795/revisions/w33795.rev1.pdf ↩
• [S10] FinCall-Surprise: A Large Scale Multi-modal Benchmark for Earning Surprise Prediction | Association for Computational Linguistics; Shu et al. | 2026-07 | https://aclanthology.org/2026.acl-long.610/ ↩
• [S11] Artificial Intelligence (AI) and Investment Fraud: Investor Alert | SEC Office of Investor Education and Advocacy, NASAA, FINRA | 2024-01-25 | https://www.investor.gov/introduction-investing/general-resources/news-alerts/alerts-bulletins/investor-alerts/artificial-intelligence-fraud ↩
• [S12] SEC Charges Rimar Capital Entities and Owner Itai Liptz for Defrauding Investors by Making False and Misleading Statements About Use of Artificial Intelligence | U.S. Securities and Exchange Commission | Announced 2024-10-10, last reviewed and updated 2024-10-11 | https://www.sec.gov/newsroom/press-releases/2024-167 ↩