Container shipping rates have started sinking. This might only be the beginning of their descent.

A steady fall in the World Container Index compiled by London-based Drewry Shipping Consultants is the latest hint that Americans—and developed world consumers more generally—might be starting to spend less on goods as fiscal stimulus drains away, inflation at a four decade high eats into wage gains, reopening economies shift back toward services and the Fed initiates aggressive rate increases. Another factor might be the lockdown in Shanghai, which appears to be disrupting the flow of goods out of China—meaning less need for container shipping.

The World Container Index is down 16% since the beginning of the year, with key routes Shanghai to Los Angeles and Shanghai to New York down 17% and 16% respectively. But most striking is the steep fall since March 10—the WCI is down 13% since that date alone. That suggests that either spring retail sales data in the U.S. will hold some nasty surprises or that the large Covid-19 outbreak in China, which really started to gain steam around mid-March, is already having an even bigger impact on global supply chains than many appreciate.

Shipping companies have started employing “void sailings,” where ships continue to sail but don’t load or discharge containers at ports, to reduce their supply on offer, and therefore support rates, according to George Griffiths, who covers the global container shipping market at S&P Global Commodity Insights.

While the WCI is still far higher than it was before the pandemic, the trajectory will likely remain downward with significant new ship supply setting sail by the end of this year. The boom in container rates over the last two years was driven by the switch away from services toward goods owing to the pandemic and government stimulus. But if Americans really are beginning to tighten their belts now, shipping rates might only see a modest pickup when China’s lockdowns end.

The cost to ship bulk goods such as grain—which are facing more disruption from the war in Ukraine—has also fallen since mid-March. But the Baltic Dry Index, a barometer of that part of the shipping world, has performed better overall in 2022. It is still roughly flat for the year.

Container shipping stocks meanwhile, are a mixed bag. A.P. Moller-Maersk is down about 20% since mid-March. Evergreen Marine and Cosco, which also operates other types of vessels, have also struggled but fared somewhat better. If China’s lockdowns don’t ease soon and U.S. consumers don’t step up to bat, investors in the sector, and perhaps more broadly in retail, may ultimately wish they had abandoned ship earlier.

Write to Megha Mandavia at megha.mandavia@wsj.com