NASA Goddard Space Flight Center
The trend and detrend are frequently encountered terms in data analysis. Yet, there is no precise mathematical definition for the trend in a data set, even though in many applications, such as financial and climatologic data analyses for example, the trend is precisely the quantity we want to find. In other applications, such as in computing the correlation function and spectral analysis, one would have to remove the trend from the data, or detrend, lest the result would be overwhelmed by the DC terms. Therefore, detrend is a necessary step before meaningful results can be obtained. As there is a lack of precise definition for the trend, detrend is also a totally ad hoc operation. In most cases, trend is taken as the result of a moving mean, a regression analysis, a filtered operation or simple curve fitting with an a priori functional form. Yet such a trend is determined subjectively and with certain idealized assumptions. Furthermore, the trend so determined is usually different from the quantity taken away in the detrend operation, which usually consists of setting the data to a simple linear fit of the data as the zero reference.
The real trend should have the following properties: First, the trend should be an intrinsic property of the data. In other words, it should be part of the data, and driven by the same mechanisms that generate the observed or measured data. Unfortunately, most of the available methods define trend by using an extrinsic approach, such as pre-selected simple functional forms. Being intrinsic, therefore, requires that the method used in defining the trend have to be adaptive. Second, the trend exists only within a given data span; therefore, it should be local, and, be associated with a local scale of data length. Consequently, the trend can only be valid within that part of data, which should be shorter than a full local wavelength. Thus, with this definition, we can avoid the difficulty encountered by most economists: “one economist’s ‘trend’ can be another’s ‘cycle.’” New definition of the trend and variability (or volatility), based on Hilbert- Huang Transform, will be given and analysis of Climate data as well as NASDAQ data will be used as examples to demonstrate the application of the Hilbert-Huang Transform.