BLOG

Behind the numbers.

Code examples, market analysis, and data quality deep-dives.

How Concentrated Are S&P 500 Earnings? Point-in-Time Index Analysis in Python
Does Volatility Scale With the Square Root of Time? Variance Ratio Test in Python
Does Goodwill Distort the Price-to-Book Screen? Goodwill-Adjusted Valuation in Python
How Are Shares Outstanding Reported (and Why They Disagree)
Do Low-Volatility Stocks Deliver Better Risk-Adjusted Returns? S&P 500 Quintile Sorts in Python
Does Trend Following Beat Buy and Hold? Time-Series Momentum in Python
Has the Stock-Bond Correlation Flipped? 60/40 Portfolio Risk in Python
What API to Use for a Stock Screener
Which Assets Hedge Inflation Shocks? Macro Factor Betas in Python
Does Covariance Shrinkage Beat the Sample Covariance? Minimum-Variance Portfolios in Python
Do Faster Inventory Turns Mean Thinner Margins? Gross Margin Return on Inventory in Python
SEC EDGAR API vs Fundamentals API: Which to Use
Does Fast Earnings Growth Persist? Rank Correlation Analysis in Python
Split Adjustment Explained: Adjusted Close vs Close
Which Trading Day of the Month Pays Best? Turn-of-the-Month Analysis in Python
Can You Use Yahoo Finance Data Commercially?
How Many Independent Bets Does a Nine-Sector Portfolio Give You? Eigenvalue Analysis in Python
Which Volatility Forecast Wins One Month Ahead? HAR vs EWMA in Python
How Concentrated Are Institutional Equity Portfolios? Form 13F Concentration Analysis in Python
Do Stocks Earn Their Returns Overnight or Intraday? Return Decomposition in Python
When Do Corporate Insiders Actually Trade? Form 4 Timing Analysis in Python
Data Requirements for Backtesting a Trading Strategy
What Is Survivorship Bias in Backtesting?
Do High Dividend Yields Come From Bigger Payouts or Falling Prices? Yield Decomposition in Python
Do Stocks Fall Harder Than They Rise? Downside Beta vs Upside Beta in Python
Does Volatility Targeting Improve Sharpe Ratios? Seven-Asset Backtest in Python
Free Stock Market Data APIs: What You Actually Get
How to Give an LLM Financial Data With an MCP Server
Does Post-Earnings Announcement Drift Survive Real Filing Dates? PEAD Event Study in Python
Do Insider Buying Clusters Predict Returns? Signal Testing in Python
Does the S&P 500 Index Effect Still Exist? Event Study in Python
Are Companies Leaving the S&P 500 Faster Than They Used To? Index Survival Analysis in Python
Does Gross Profitability Predict Stock Returns? Quintile Factor Test in Python
What Growth Rate Is the Market Pricing In? Reverse DCF in Python
Does a Strong Balance Sheet Cushion Drawdowns? Leverage and Downside Risk in Python
Does Ticker Recycling Corrupt a Mean-Reversion Backtest? Entity-Resolved Z-Scores in Python
How Much of a Growth Screen's Backtested Edge Is Survivorship Bias? Point-in-Time Index Testing in Python
Why Do Leveraged ETFs Decay? Measuring Volatility Drag in Python
Are Consumer Staples Margins Shrinking Under Inflation? Gross Margin Trend Analysis in Python
Do Weak Jobs Reports Predict Market Drawdowns? NFP Surprise Event Study in Python
Is the Rotation From Tech to Industrials Backed by Earnings? Relative EPS Growth Analysis in Python
Is the Semiconductor Rally Broadening Beyond NVIDIA? Return Dispersion Analysis in Python
Which Stocks Benefit Most When Oil Prices Fall? Oil Beta Screening in Python
Do Bond Returns Predict Stock Returns? Granger Causality Test in Python
Which Stocks Actually Drive Portfolio Returns? Shapley Value Attribution in Python
Does "Sell in May" Still Work? Calendar Anomaly Backtest in Python
How to Build Complete Price History Through Ticker Changes? Entity Resolution in Python
Are KO and PEP Cointegrated? Pairs Trading Signal Construction in Python
Which Commodities Have the Strongest Momentum? Rotation Backtest in Python
Which Commodity ETFs Have the Worst Tail Risk? Expected Shortfall in Python
Are Gold Miners Leveraged Gold Bets? Rolling Beta Analysis in Python
Does the Base-Metals-to-Gold Ratio Lead Cyclical Stocks? Signal Test in Python
Can Risk Parity Tame Commodity Volatility? Portfolio Optimization in Python
Are Power Stocks Becoming an AI Infrastructure Trade? Momentum Screening in Python
Which AI Chip Stocks Have Margin Momentum? Profitability Trend Analysis in Python
Which AI Stocks Are Cheapest Relative to Growth? Growth-Adjusted Valuation in Python
Does AI Stock Leadership Persist? Momentum Backtest in Python
Which AI Stocks Have the Cleanest Balance Sheets? Net Cash Screening in Python
Can Risk Parity Reduce Mega-Cap Drawdowns? Portfolio Optimization in Python
Which Growth Stocks Are Self-Funding? Cash-Flow Quality Screening in Python
Which Sectors Struggle When the Dollar Rallies? Sector Rotation Analysis in Python
Do Cheap Stocks Hold Up When Bonds Sell Off? Valuation Rotation in Python
Does the Nasdaq 100 Have Better Growth Quality Than the Dow? Index Constituent Analysis in Python
Do Healthcare Cash-Flow Margins Predict Returns? Signal Evaluation in Python
Which Dividend Stocks Survive a Cash-Flow Stress Test? Dividend Screening in Python
Does Heavy Insider Selling Predict Weak Returns? Insider Flow Test in Python
Can Quality Screens Reduce Small-Cap Balance-Sheet Risk? Russell 2000 Test in Python
Which Retailers Have Positive Operating Leverage? Margin Screening in Python
Is MSTR a Leveraged Bitcoin Proxy? Rolling Beta Analysis in Python
Is Micron's Memory Cycle Recovering? Inventory and Margin Forecasting in Python
Which Sectors Work When Bonds Rally? Rate-Sensitive Rotation in Python
Do One-Month Price Extremes Reverse? Signal Evaluation in Python
Do Low-Volatility S&P 500 Stocks Reduce Drawdowns? Factor Test in Python
Is AI Capex Paying Back Fast Enough? Revenue Hurdle Forecasting in Python
Could Shorter AI Asset Lives Hit Earnings? Depreciation Stress Test in Python
How Much AI Capex Risk Can a Portfolio Remove? Constrained Optimization in Python
Is the AI Capex Trade Crowded? Rolling Volatility and Sector Rotation in Python
Did the AI Boom Come From Existing S&P 500 Members? Point-in-Time Momentum Test in Python
Is AI Revenue Circular? Customer-Vendor Capex Loop Analysis in Python
Is the AI Trade Connected to Private Credit? Rolling Correlation Network in Python
Is Apollo More Balance-Sheet Sensitive Than Peers? Leverage Screen in Python
Are AI Earnings Supported by Cash Flow? Accrual and Capex Screen in Python
Can Defensive Stocks Hedge AI Drawdowns? Basket Regime Test in Python
How Fast Does the Market Price In Fed Decisions? FOMC Event Study in Python
How Much Are Options Sellers Overpaid? The Variance Risk Premium in Python
Which Companies Have the Worst Earnings Quality? Sloan Accrual Screen with Geographic Revenue Data in Python
Does the Oil-to-Gold Ratio Signal Recessions? XLE/GLD Backtest in Python
Is AI Spending Crowding Out Free Cash Flow? Capex Sustainability Across the Mag 7 in Python
Does a Long Energy / Short Bonds Portfolio Capture Inflation Surprises? Factor Construction in Python
Can a Hidden Markov Model Detect Oil Market Regimes? HMM Analysis in Python
Do Grain Prices Predict Food Inflation? Granger Causality Test in Python
Does the Corporate Credit Spread Predict Stock Market Crashes? BAA-AAA Spread Analysis in Python
Do Oil Stocks Hedge Inflation? Rolling Beta Analysis in Python
Which Stocks Are Most Rate-Sensitive? Equity Duration via Bond Beta in Python
Which Companies Have the Highest Accrual Ratios? Earnings Quality Screening in Python
Is Alpha Persistent or Decaying? Rolling Sharpe Ratio Analysis in Python
Are Markets Trending or Mean-Reverting? Hurst Exponent Analysis in Python
Is Consumer Discretionary vs Staples a Leading Indicator? XLY/XLP Ratio Analysis in Python
Does Heavy Capex Predict Future Stock Returns? Capital Expenditure Analysis in Python
How to Estimate Cost of Equity Using CAPM in Python
Is Volatility Predictable? Testing for Volatility Clustering in Python
Which Industrials Are Overleveraged? Net Debt to EBITDA Screening in Python
GM Before and After Bankruptcy: Why Entity Resolution Matters for Financial Data
What Is Adjusted Beta? Merrill Lynch Beta Shrinkage in Python
How Good Is a Stock Pick? Information Ratio and Tracking Error in Python
Do Stock Returns Follow a Normal Distribution? Testing for Fat Tails in Python
Which Large Caps Have the Highest Free Cash Flow Yield? FCF Screening in Python
Which Sectors Won Over 5 Years? Sector Rotation Analysis in Python
How to Forecast Stock Volatility with GARCH Models in Python
Are Stock Prices Mean-Reverting? Augmented Dickey-Fuller Test in Python
How to Calculate CAPM Alpha and Beta with Regression in Python
How to Compare Sector Sharpe Ratios and Sortino Ratios in Python
DELL: Why Stitching Historical Price Data Together Is Wrong
How to Analyze Drawdown and Recovery for Bank Stocks in Python
How to Screen SaaS Stocks by Revenue Growth and Cash Flow in Python
How to Screen REITs by Dividend Yield and Valuation in Python
How Correlated Are the Magnificent 7? Intra-Group Correlation in Python
AAPL vs XOM: Do Individual Stocks Have Seasonal Patterns?
How to Rank Large-Cap Stocks by Momentum in Python
How to Build a Multi-Endpoint Financial Dashboard in Python
How to Compare Volatility Across Energy Stocks in Python
How to Screen Healthcare Stocks by Valuation in Python
How to Build a Sector Correlation Matrix for Portfolio Diversification in Python
How to Find Oversold and Overbought Stocks Using Z-Scores in Python
How to Measure Earnings Quality: Cash Flow vs Net Income in Python
How to Build a Multi-Factor Stock Screen in Python (Value + Momentum + Quality)
How to Build a Simple DCF Model for Any Stock in Python
How to Screen Tech Stocks by Revenue Growth in Python
How to Screen Stocks by Balance Sheet Health in Python
Is "Sell in May" Real? SPY Monthly Seasonality Over 10 Years
How to Compare Sector Performance YTD Using Python
How to Screen Dividend Stocks by Yield and Quality in Python
How to Calculate Max Drawdown and Recovery Time for Any Stock in Python
How to Compare Profitability Across Mega-Cap Tech Stocks in Python
Why Ticker Symbols Are Unreliable: The Recycling Problem Every Quant Should Know
How to Calculate and Compare Stock Volatility in Python
How to Screen Blue-Chip Stocks by P/E Ratio in Python
How to Track Companies Through Ticker Changes, Bankruptcies, and Renames in Python
S&P 500 Turnover: How Much the Index Has Changed Since 2010
How to Calculate Stock Beta and Correlation in Python
← All articles

How Concentrated Are S&P 500 Earnings? Point-in-Time Index Analysis in Python

What's the question?

Concentration in the S&P 500 is normally discussed in terms of market value, and the ten largest members now carry roughly a third of the index weight. Market value is a price, though, and a price is an opinion about future profits that can be revised overnight.

Reported profit is different. It arrives in an audited annual filing and cannot be revised by sentiment. So the question is whether the profit behind the index has concentrated as fast as the price. If the ten largest earners still produce the share of aggregate profit they produced in 2010, rising index concentration is a valuation phenomenon and should mean-revert like one. If their share has climbed too, the concentration is structural.

Two measures answer this: the top-ten share of the profit pool, being the sum of positive net income across members, and the effective number of companies, the reciprocal of the Herfindahl index of profit shares. The second reports how many equally sized earners would produce the observed concentration.

The approach

Any comparison across sixteen years falls apart if the roster is wrong. Today’s membership list is a list of survivors, and survivors were disproportionately profitable, so backdating it decides the answer in advance.

  1. Pull S&P 500 membership at each 31 December from 2010 to 2025, keyed to entity identifiers rather than tickers. A ticker is a lease rather than a name: S, PX and SLE carried Sprint, Praxair and Sara Lee in 2010 and carry SentinelOne, P10 and Super League Enterprise today.
  2. Pull one annual income statement per member per roster. Fiscal year Y covers period ends from June of Y to May of Y+1, which places Microsoft’s June year end and Walmart’s January year end in the year they describe.
  3. Screen the panel. A member without both figures for that fiscal year, or without positive revenue, leaves that year’s sample. Where a fiscal-year change yields two statements inside one window, the later is kept.
  4. Compute both measures for revenue and for net income separately, since revenue is a scale measure and profit a margin measure, then recompute fiscal 2010 using today’s roster to size what survivorship bias does here.

Coverage runs from 467 of 486 members in fiscal 2010 to 485 of 496 in fiscal 2025.

Code

import numpy as np
import pandas as pd
import xfinlink as xfl

xfl.set_api_key("YOUR_API_KEY")  # free at https://xfinlink.com/signup

def roster(as_of):
    df = xfl.index("sp500", as_of=as_of).dropna(subset=["entity_id"])
    return sorted(set(df["entity_id"].astype(int)))

def fiscal_year(ids, year):
    frames = [xfl.fundamentals(entity_id=ids[i:i + 100], period_type="annual",
                               start=f"{year}-06-01", end=f"{year + 1}-05-31",
                               fields=["revenue", "net_income"], max_rows=100000)
              for i in range(0, len(ids), 100)]
    f = pd.concat([x for x in frames if len(x)], ignore_index=True)
    f = f.dropna(subset=["revenue", "net_income"])
    f = f[f["revenue"] > 0]
    return f.sort_values("period_end").groupby("entity_id", as_index=False).tail(1)

def concentration(values):
    pos = np.sort(np.asarray(values, dtype=float)[np.asarray(values) > 0])[::-1]
    w = pos / pos.sum()
    return 100 * w[:10].sum(), 1.0 / np.sum(w ** 2)

for year in range(2010, 2026):
    f = fiscal_year(roster(f"{year}-12-31"), year)
    rev_top10, rev_neff = concentration(f["revenue"])
    ni_top10, ni_neff = concentration(f["net_income"])
    print(f"{year}  revenue {rev_top10:.1f}% / {rev_neff:.1f}   "
          f"profit {ni_top10:.1f}% / {ni_neff:.1f}")

Full script with formatting and visualisation: sp500-earnings-concentration-point-in-time-python.py

Output

Two panel chart of S&P 500 concentration from 2010 to 2025: the top-ten share of profit rises from 23.4 to 35.9 percent while the revenue share is flat, and the effective number of profit generators falls from 98 to 56
S&P 500 earnings vs revenue concentration, point-in-time rosters
FY    members  covered  rev top10  rev Neff  NI top10  NI Neff  profit pool  loss-makers
2010      486      467      21.8%     102.9     23.4%     98.3         757b           25
2011      488      468      22.0%      99.9     25.7%     90.4         862b           27
2012      491      470      21.3%     103.0     27.6%     79.3         845b           36
2013      491      470      21.1%     105.4     23.4%    101.8         979b           18
2014      492      469      20.8%     107.9     23.6%    103.6         959b           20
2015      495      475      20.7%     110.9     23.3%     97.2         972b           41
2016      496      483      20.8%     111.6     22.4%    103.7         987b           47
2017      498      486      21.0%     112.0     26.7%     89.1       1,080b           42
2018      499      490      21.6%     110.1     22.6%    104.1       1,213b           28
2019      500      493      22.0%     109.3     27.2%     79.1       1,301b           33
2020      500      493      23.4%      99.5     29.0%     76.8       1,107b           79
2021      500      494      23.7%     102.4     28.3%     76.8       1,840b           20
2022      499      497      23.7%     101.9     27.5%     80.1       1,688b           40
2023      499      499      24.5%      99.3     30.9%     70.5       1,813b           32
2024      499      495      25.2%      95.5     34.8%     63.3       1,966b           24
2025      496      485      25.4%      93.9     35.9%     56.1       2,193b           26

Top 10 earners, FY2010 vs FY2025 ($m)
#  FY2010      net income   FY2025      net income
1  XOM             30,460   GOOGL          132,170
2  T               19,864   NVDA           120,067
3  CVX             19,024   AAPL           112,010
4  MSFT            18,760   MSFT           101,832
5  JPM             17,370   AMZN            77,670
6  WMT             16,389   BRK             66,968
7  IBM             14,833   META            60,458
8  AAPL            14,013   JPM             57,048
9  JNJ             13,334   BAC             30,509
10 BRK             12,967   XOM             28,844

FY2010 measured two ways
point-in-time 2010 roster : 467 names, top-10 share 23.4%, Neff 98.3
today's roster backdated  : 456 names, top-10 share 24.5%, Neff 88.2

What this tells us

Revenue barely moved. The ten largest sellers took 21.8 percent of index revenue in fiscal 2010 and 25.4 percent in fiscal 2025, and the effective number of revenue generators fell only from 102.9 to 93.9. Profit moved a long way: the top-ten share went from 23.4 percent to 35.9 percent and the effective count fell from 98.3 to 56.1, a decline of 43 percent. Since the revenue base did not concentrate, the gap is margin. Alphabet converted 32.8 percent of its fiscal 2025 revenue into net income and NVIDIA converted 55.6 percent, against 10.8 percent at Amazon, the largest seller in the index.

Net income absorbs impairments, settlements and tax charges, so the profit series is noisy, and the effective count swings between 76 and 104 through the middle of the decade without trending. What follows fiscal 2022 differs in kind: three consecutive falls, from 80.1 to 70.5 to 63.3 to 56.1, with no reversal.

Measuring fiscal 2010 with today’s members returns a top-ten share of 24.5 percent instead of 23.4 percent and an effective count of 88.2 instead of 98.3, because the companies that later left the index were mid-sized earners whose absence tightens the distribution. That biased start shrinks the measured decline in effective count from 42.2 companies to 32.1, understating the change by 24 percent.

So what?

An index-level earnings forecast is mostly a forecast about ten companies and should be built that way. A bottom-up model aggregating 500 independent estimates assigns weight to names that no longer move the total, and its error is dominated by whichever of the five largest earners misses.

The result also constrains the bubble argument. Price concentration that outruns profit concentration will unwind on sentiment; price concentration that tracks it will not, and both have risen here.

For equal-weight investors the number to watch is the effective count rather than the top-ten weight. An equal-weighted S&P 500 holds roughly 0.2 percent in each of the ten companies producing 35.9 percent of the profit, an underweight to the earnings base that should be sized rather than inherited. And rebuild the roster point in time: the shortcut costs about a quarter of the effect here, always in the direction that makes the past resemble the present.

Built with xfinlink — free financial data API for Python. pip install -U xfinlink

Built with xfinlink — free financial data API for Python. pip install -U xfinlink
← All articles