Why alt data is better for short-term investment forecasting

Internet data continues its rapid pace of growth as more users come online to conduct business transactions, upload personal information, and track economic activity. According to a Statista report, the total amount of data created, captured, copied, and consumed reached 64.2 zettabytes (64.2 trillion gigabytes) in 2020 and is projected to more than triple to 180 zettabytes by 2025.

E-commerce activity, financial and economic reports, satellite images, and other information pertaining to investment valuations make up a large portion of this data. Portfolio managers looking to beat the market have taken advantage of this information when making portfolio decisions, giving rise to the term "alternative data" to describe these new sources of investment intelligence.

Additionally, a new, promising source of investment intelligence is ESG data. Most of it can be classified under alternative data as there’s still a lack of widespread standardization. A lot of investment intelligence is, as such, collected from news reports, social media posts regarding ESG, and even satellite imagery.

Alternative data goes beyond traditional sources of information such as company annual reports and economic forecasts that are released on a yearly, monthly, or weekly basis. Since the internet is always active, alternative data provides timelier, more exclusive, and granular data that transforms how investment professionals forecast future outcomes and make investment decisions.

How Investors Use Alternative Data For Forecasting

The most common alternative data sources include web traffic, credit card transactions, point-of-sale transactions, geolocation data, and satellite imagery. However, the utility of alternative data depends significantly on several factors that include its source, time of extraction, and quality of extraction. In this regard, not all alternative data is equally valuable for forecasting.

Alternative data types can also vary significantly within sectors across a diverse range of industries. Some examples include:

• Real-estate: current offerings, rental availability, home prices, local business reviews, building permits, energy consumption
• E-Commerce: Current product information, SEO trends, customer comments, and reviews, social media sentiment
• Business data: Employee turnover, job postings, information from company registers, employee satisfaction scores, mergers and acquisitions intelligence
• Commodities: satellite images of oil tankers, agricultural activity, mining projects

Use of Alt Data for Short vs. Long-Term Forecasting

In most cases, long-term forecasting is typically more complicated than short-term forecasting due to increasing variables that frequently change over time. Collecting data relevant to long-term forecasting is also more challenging, requiring more time and analysis to assess relevant factors associated with making a solid prediction.

This "horizon" effect was examined in a recent paper from the Swiss Finance Institute, where researchers studied how alternative data affects the informativeness of both short and long-term financial forecasts. Given that alternative data is typically less cost-intensive in the short term, researchers hypothesized that forecasters allocate more effort to collecting short-term information at the expense of long-term data collection, resulting in better short-term forecasts.

To demonstrate this effect, researchers analyzed a large sample of third-party short and long-term forecasts that used data from Stocktwits -- a data aggregator that captures social media mentions of stocks, crypto, futures, and forex. They found that the opportunity cost of collecting long-term data led analysts to focus on short-term alternative data, and concluded that the availability of these data sources leads to more effective short-term forecasts.

Other Alternative Data Use Cases

Use cases for short-term forecasts using alternative data continue to grow as the amount of online data increases. Some current use cases specific to short-term forecasting include:

Export Trackers

Analysts at Goldman Sachs created an export tracker that gave them a more direct read on the value of exports on a much shorter timeline. When compared to traditional export reports published by government agencies, the export tracker is a more direct source of data that provides analysts with more current information. In addition, the dataset offers greater geographic variation than traditional datasets that are typically limited by the governing bodies that produce the reports.

In addition, export trackers offer greater granularity that isolates the impacts of economic shocks, giving analysts insights into how sudden economic shifts impact areas of the economy.

Construction Satellite Imagery

Investment managers use satellite images to obtain information on the construction sector, giving them insight on the number of projects and investments in agricultural machinery. Machine learning is also being applied in this case to provide new and unconventional input for economic forecasting models.

Credit Card Spending

Datasets provided by credit card companies provide critical data into what sectors are growing and declining. Besides giving insight into demand factors, these datasets also offer demographic information that includes customer ages, income levels, locations, and time periods with the highest spending.

Online Investment Communities

Online investment groups like the WallStreetBets community on the Reddit social media platform help investors predict speculative increases in the prices of stocks, commodities, and other investment instruments. This information is typically obtained through web scraping, a process that extracts data from public websites.

How Web Scraping Works

Web scraping uses scripts or bots to scour websites and extract data. These scripts are designed to read HTML and extract the relevant data, where it is then converted or "parsed" into a format that analysts can read. To prevent server issues, scrapers use AI and ML-powered proxies to distribute requests and maintain anonymity during the extraction process.

Web scraping can also be used to produce other alternative data types, including social media insights, product pricing and stock information, SEO keyword data, and business information from online directories.

Photo Credit: Andy Dean Photography/Shutterstock

Gediminas Rickevičius is VP of Global Partnerships & Business Development at Oxylabs.io. If you want to find out more about how alternative data brings value to investors you can download our free white paper Why Is Alternative Data Playing an Increasingly Larger Role in Financial Markets?