Applying machine learning algorithms to predict the stock price trend in the stock market - The case of Vietnam

The study addresses the long-standing challenge of predicting short-term movements in stock prices, a task complicated by market complexity and diverse information sources. While efficient market and random walk hypotheses argue against predictability, a growing body of research suggests partial predictability using data-driven methods, including machine learning and deep learning. The authors posit that recurrent neural networks, particularly LSTM, with their ability to capture short- and long-term dependencies in time series, can effectively forecast short-term stock price trends. The research gap identified is the limited application and testing of LSTM combined with technical indicators in Vietnam’s stock market. The objective is to evaluate the applicability and performance of LSTM models, augmented by technical analysis indicators (SMA, MACD, RSI), in predicting price trends for VN-Index and VN30 stocks.

Theoretical context contrasts the Efficient Market Hypothesis (Fama, 1970) and Random Walk Hypothesis (Malkiel) with evidence of partial predictability documented across economics, statistics, and computer science. Technical analysis, based on price and volume and assumptions of trends and repeating patterns, has shown mixed but notable effectiveness (e.g., moving average rules, MACD, RSI), though subject to data bias. Recurrent Neural Networks (RNNs) and specifically LSTMs address vanishing/exploding gradient issues via gated memory cells (Hochreiter & Schmidhuber, 1997), making them suitable for sequence/time-series tasks. Prior studies have applied neural networks and LSTMs to financial time series, sometimes integrating textual sentiment or event data, and report strong short-term predictive performance (e.g., Nelson et al., 2017; Chen et al., 2015; Di Persio & Honchar, 2016; Zhuge et al., 2017; Mehtab & Sen, 2019, 2020). The literature indicates LSTM’s stability and efficiency for short-horizon stock prediction and supports combining technical indicators with machine learning models.

Data: Historical daily data for VN-Index and 30 large-cap, liquid VN30 stocks (31 series total) were collected from vietstock.vn from each listing date through April 1, 2021. Variables: open, high, low, close, and volume. Industry classification follows GICS. Preprocessing steps: (1) Data quality checks and corrections for missing/erroneous entries. (2) Computation of technical indicators per stock: Simple Moving Average (SMA), Moving Average Convergence Divergence (MACD), and Relative Strength Index (RSI). (3) Merge price series with indicators; drop observations lacking indicator values due to lookback requirements. (4) Use the merged dataset as inputs to the LSTM. Train-test split: Training data span from listing start date to 12/31/2020; test set spans 01/01/2021–04/01/2021 (78 trading sessions). Sets are independent. Model: LSTM sequence model with a lookback (step) window of 60 days to predict the next day’s price. Architecture (Sequential): four LSTM layers with ReLU activations and Dropout(0.1): Layer1 units=30; Layer2 units=40; Layer3 units=50; Layer4 units=60; followed by Dense(1). Implementation used Python (Sklearn, Keras, TensorFlow). Compilation: optimizer=Adam; loss=mean_squared_error. Training: epochs=1000; batch_size=32; EarlyStopping(monitor='loss', patience=8, restore_best_weights=True). Evaluation: The model forecasts on the test period were compared to actual closing prices. The paper defines an accuracy measure A_j = (1/n) Σ_i |P_ij − V_ij| and compares model performance to a 93% baseline derived from a 7% maximum daily price fluctuation rule on the Ho Chi Minh Stock Exchange; models performing below 93% are deemed inefficient. Performance is reported per ticker.

- The LSTM model’s predicted price trends closely track actual test-set movements for many tickers; visual comparisons show small deviations during 78 test sessions (Jan 1–Apr 1, 2021). - Accuracy exceeded the 93% baseline for most stocks studied. - Highest reported accuracy: PNJ at 97.7%. Stocks such as MSN and TPB reached approximately 97%. - Lower-performing cases included NVL at 78.9%, TCH at 86.8%, and approximately 89% for FPT, PLX, POW, VPB, and VRE. - VN-Index forecasts also showed close alignment between predicted and actual prices in the test period. Overall, results support the appropriateness of LSTM for short-term stock price forecasting in Vietnam, particularly when combined with technical indicators.

The findings answer the research question by demonstrating that an LSTM-based approach leveraging historical prices and technical indicators can achieve high short-term predictive accuracy for a majority of VN30 stocks, surpassing a practical baseline. The close match between predicted and realized prices suggests the LSTM’s capacity to capture temporal dependencies and patterns embedded in price series and indicators. Variability across tickers (e.g., lower accuracy for NVL, TCH) indicates that sudden structural changes or higher volatility may challenge the model, highlighting the importance of data characteristics. The results reinforce prior literature on LSTM effectiveness for financial time series and suggest that integrating technical indicators adds informative signals. These outcomes are relevant for investors and regulators seeking data-driven tools for short-term forecasting in emerging markets.

The study shows that LSTM models, augmented with technical analysis indicators (SMA, MACD, RSI), provide strong short-term predictive performance for VN-Index and most VN30 stocks, supporting their suitability for financial time series forecasting in Vietnam. The approach aligns with and extends prior evidence of LSTM effectiveness. Future research directions include: (1) exploring alternative and ensemble machine learning methods (e.g., Random Forests, SVMs, hybrid models) to potentially improve performance; (2) incorporating unstructured data (news text, audio, images) and market sentiment; and (3) broadening datasets to encompass additional Vietnamese exchanges to test robustness and generalizability.

- Market scope limited to Ho Chi Minh City’s stock market (VN-Index and VN30 constituents), which may constrain generalizability across Vietnam or other markets. - Model performance may degrade during periods of strong price fluctuations; emerging-market idiosyncrasies (small float, market manipulation, legal risks) can impede predictability. - The evaluation relies on a specific accuracy definition and a baseline derived from daily price limit rules; alternative error metrics and benchmarks could yield different assessments. - Only structured numerical data were used; exclusion of unstructured data (e.g., news/sentiment) may omit influential information.