A Comprehensive Review of Stock Price Prediction Using Text Mining

Hossein Hassani; Maedeh TajMazinani; Reza Raei

doi:10.47654/v26y2022i2p116-152

Authors

Hossein Hassani University of Tehran, Webster University Corresponding Author
Maedeh TajMazinani Author
Reza Raei Author

DOI:

https://doi.org/10.47654/v26y2022i2p116-152

Keywords:

Stock price prediction, Sentiment analysis, Text mining, Big data

Abstract

Purpose. In several research studies, stock price prediction has been explored, and sentiment analysis has been identified as essential for predicting stock price behavior. The availability of news and social media networks and the rapid development of natural language processing methods attracted many researchers to this field. However, there is a rare comprehensive framework for approaching the issue. As an interdisciplinary problem consisting of behavioral-economic and artificial intelligence, discussing previous studies clarify the path ahead for upcoming research.

Design/methodology/approach. This paper aims at promoting the existing literature in this field by focusing on different aspects of previous studies and presenting an explicit picture of their components. We, furthermore, compare each system with the rest and identify their main differentiating factors. This paper summarized and systematized studies that seek to predict stock prices based on text mining and sentiment analysis in a systematic review paper.

Findings. The results showed state-of-the-art algorithms and pre-processing approaches. It also discussed the developments made during recent years and addressed the existing gap in this field to the research community.

References

Adebiyi, A. A., Ayo, C. K., Adebiyi, M., & Otokiti, S. O. (2012). Stock price prediction using neural network with hybridized market indicators. Journal of Emerging Trends in Computing and Information Sciences, 3(1).

Ahmadi, E., Jasemi, M., Monplaisir, L., Nabavi, M. A., Mahmoodi, A., & Jam, P. A. (2018). New efficient hybrid candlestick technical analysis model for stock market timing on the basis of the Support Vector Machine and Heuristic Algorithms of Imperialist Competition and Genetic. Expert Systems with Applications, 94, 21-31.

Alkubaisi, G. A. A. J., Kamaruddin, S. S., & Husni, H. (2018). Stock Market Classification Model Using Sentiment Analysis on Twitter Based on Hybrid Naive Bayes Classifiers. Computer and Information Science, 11(1), 52-64.

Audrino, F., Sigrist, F., & Ballinari, D. (2020). The impact of sentiment and attention measures on stock market volatility. International Journal of Forecasting, 36(2), 334-357.

Ballings, M., Van den Poel, D., Hespeels, N., & Gryp, R. (2015). Evaluating multiple classifiers for stock price direction prediction. Expert Systems with Applications, 42(20), 7046-7056.

Berutich, J. M., López, F., Luna, F., & Quintana, D. (2016). Robust technical trading strategies using GP for algorithmic portfolio selection. Expert Systems with Applications, 46, 307-315.

Bikas, E., Jurevičienė, D., Dubinskas, P., & Novickytė, L. (2013). Behavioural finance: The emergence and development trends. Procedia-social and behavioral sciences, 82, 870-876.

Bisoi, R., & Dash, P. K. (2014). A hybrid evolutionary dynamic neural network for stock market trend analysis and prediction using unscented Kalman filter. Applied Soft Computing, 19, 41-56.

Bollen, J., Mao, H., & Zeng, X. (2011). Twitter mood predicts the stock market. Journal of computational science, 2(1), 1-8.

Broadstock, D. C., & Zhang, D. (2019). Social-media and intraday stock returns: The pricing power of sentiment. Finance Research Letters.

Brownlee, J. (2017). Deep learning for natural language processing: develop deep learning models for your natural language problems: Machine Learning Mastery.

Cervelló-Royo, R., Guijarro, F., & Michniuk, K. (2015). Stock market trading rule based on pattern recognition and technical analysis: Forecasting the DJIA index with intraday data. Expert Systems with Applications, 42(14), 5963-5975.

Chakraborty, K., Bhattacharyya, S., Bag, R., & Hassanien, A. A. (2018). Sentiment analysis on a set of movie reviews using deep learning techniques. Social Network Analytics: Computational Research Methods and Techniques, 127.

Chen, M.-Y., Liao, C.-H., & Hsieh, R.-P. (2019). Modeling Public Mood and Emotion: Stock Market Trend Prediction with Anticipatory Computing Approach. Computers in Human Behavior.

Chen, W., Yeo, C. K., Lau, C. T., & Lee, B. S. (2018). Leveraging social media news to predict stock index movement using RNN-boost. Data & Knowledge Engineering, 118, 14-24.

Chong, E., Han, C., & Park, F. C. (2017). Deep learning networks for stock market analysis and prediction: Methodology, data representations, and case studies. Expert Systems with Applications, 83, 187-205.

da Costa, T. R. C. C., Nazário, R. T., Bergo, G. S. Z., Sobreiro, V. A., & Kimura, H. (2015). Trading system based on the use of technical analysis: A computational experiment. Journal of Behavioral and Experimental Finance, 6, 42-55.

Darsono, S. N. A. C., Wong, W.-K., Nguyen, T. T. H., Jati, H. F., & Dewanti, D. S. (2022). Good Governance and Sustainable Investment: The Effects of Governance Indicators on Stock Market Returns. Advances in Decision Sciences, 26(1), 69-101.

Das, S., Behera, R. K., & Rath, S. K. (2018). Real-time sentiment analysis of Twitter streaming data for stock prediction. Procedia computer science, 132, 956-964.

Dash, R., & Dash, P. K. (2016). A hybrid stock trading framework integrating technical analysis with machine learning techniques. The Journal of Finance and Data Science, 2(1), 42-57.

Derakhshan, A., & Beigy, H. (2019). Sentiment analysis on stock social media for stock price movement prediction. Engineering Applications of Artificial Intelligence, 85, 569-578.

Diamond, Peter A., Behavioral Economics (March 15, 2008). MIT Department of Economics Working Paper No. 08-03, Available at SSRN: https://ssrn.com/abstract=1108588 or http://dx.doi.org/10.2139/ssrn.1108588

Dickinson, B., & Hu, W. (2015). Sentiment analysis of investor opinions on twitter. Social Networking, 4(03), 62.

Ding, X., Zhang, Y., Liu, T., & Duan, J. (2015). Deep learning for event-driven stock prediction. Paper presented at the Twenty-Fourth International Joint Conference on Artificial Intelligence.

Fama, E. F. (1965). The behavior of stock-market prices. The journal of Business, 38(1), 34-105.

Fox, J., & Sklar, A. (2009). The myth of the rational market: A history of risk, reward, and delusion on Wall Street: Harper Business New York.

Franses, P. H. (2018). Prediction Intervals for Expert-Adjusted Forecasts. Advances in Decision Sciences, 22, 1-12.

Friesen, G. C., Weller, P. A., & Dunham, L. M. (2009). Price trends and patterns in technical analysis: A theoretical and empirical examination. Journal of Banking & Finance, 33(6), 1089-1100.

Geva, T., & Zahavi, J. (2014). Empirical evaluation of an automated intraday stock recommendation system incorporating both market data and textual news. Decision support systems, 57, 212-223.

Groth, S. S., & Muntermann, J. (2011). An intraday market risk management approach based on textual analysis. Decision support systems, 50(4), 680-691.

Guo, K., Sun, Y., & Qian, X. (2017). Can investor sentiment be used to predict the stock price? Dynamic analysis based on China stock market. Physica A: Statistical Mechanics and its Applications, 469, 390-396.

Hagenau, M., Liebmann, M., & Neumann, D. (2013). Automated news reading: Stock price prediction based on financial news using context-capturing features. Decision support systems, 55(3), 685-697.

Hassani, H., Beneki, C., Unger, S., Mazinani, M. T., & Yeganegi, M. R. (2020). Text Mining in Big Data Analytics. Big Data and Cognitive Computing, 4(1), 1.

Hon, T.-Y., Moslehpour, M., & Woo, K.-Y. (2021). Review on behavioral finance with empirical evidence. Advances in Decision Sciences, 25(4), 1-30.

Ji, X., Wang, J., & Yan, Z. (2021). A stock price prediction method based on deep learning technology. International Journal of Crowd Science.

Kalyani, J., Bharathi, P., & Jyothi, P. (2016). Stock trend prediction using news sentiment analysis. arXiv preprint arXiv:1607.01958.

Kao, L.-J., Chiu, C.-C., Lu, C.-J., & Yang, J.-L. (2013). Integration of nonlinear independent component analysis and support vector regression for stock price forecasting. Neurocomputing, 99, 534-542.

Katayama, D., & Tsuda, K. (2018). A Method of Measurement of The Impact of Japanese News on Stock Market. Procedia computer science, 126, 1336-1343.

Kazem, A., Sharifi, E., Hussain, F. K., Saberi, M., & Hussain, O. K. (2013). Support vector regression with chaos-based firefly algorithm for stock market price forecasting. Applied Soft Computing, 13(2), 947-958.

Koyel Chakraborty, S. B., Rajib Bag, Aboul Alla Hassanien. (2019). social network analytics: academic press.

Kristian Henrickson, F. R., Francisco Camara Pereira. (2019). Mobility Patterns, Big Data and Transport Analytics: Tools and Applications for Modeling.

Kudryavtsev, A. (2020). Immediate And Longer-Term Stock Price Dynamics Following Large Stock Price Changes. Annals of Financial Economics, 15(01), 2050002.

Lee, C. S., Cheang, P. Y. S., & Moslehpour, M. (2022). Predictive Analytics in Business Analytics: Decision Tree. Advances in Decision Sciences, 26(1), 1-29.

Li, Q., Wang, T., Li, P., Liu, L., Gong, Q., & Chen, Y. (2014). The effect of news and public mood on stock movements. Information Sciences, 278, 826-840.

Li, X., Wu, P., & Wang, W. (2020). Incorporating stock prices and news sentiments for stock market prediction: A case of Hong Kong. Information Processing & Management, 102212.

Liu, B. (2020). Sentiment analysis: Mining opinions, sentiments, and emotions: Cambridge university press.

Liu, P., Xia, X., & Li, A. (2018). Tweeting the financial market: Media effect in the era of Big Data. Pacific-Basin Finance Journal, 51, 267-290.

Liu, Y., Qin, Z., Li, P., & Wan, T. (2017). Stock volatility prediction using recurrent neural networks with sentiment analysis. Paper presented at the International Conference on Industrial, Engineering and Other Applications of Applied Intelligent Systems.

Maqsood, H., Mehmood, I., Maqsood, M., Yasir, M., Afzal, S., Aadil, F., . . . Muhammad, K. (2020). A local and global event sentiment based efficient stock exchange forecasting using deep learning. International Journal of Information Management, 50, 432-451.

Mizuno, H., Kosaka, M., Yajima, H., & Komoda, N. (1998). Application of neural network to technical analysis of stock market prediction. Studies in Informatic and control, 7(3), 111-120.

Moat, H. S., Curme, C., Avakian, A., Kenett, D. Y., Stanley, H. E., & Preis, T. (2013). Quantifying Wikipedia usage patterns before stock market moves. Scientific reports, 3, 1801.

Mostafa, M. M. (2013). More than words: Social networks’ text mining for consumer brand sentiments. Expert Systems with Applications, 40(10), 4241-4251.

Murphy, J. J. (1999). Technical analysis of the financial markets: A comprehensive guide to trading methods and applications: Penguin.

Nassirtoussi, A. K., Aghabozorgi, S., Wah, T. Y., & Ngo, D. C. L. (2014). Text mining for market prediction: A systematic review. Expert Systems with Applications, 41(16), 7653-7670.

Nazário, R. T. F., e Silva, J. L., Sobreiro, V. A., & Kimura, H. (2017). A literature review of technical analysis on stock markets. The Quarterly Review of Economics and Finance, 66, 115-126.

Nguyen, T. H., Shirai, K., & Velcin, J. (2015). Sentiment analysis on social media for stock movement prediction. Expert Systems with Applications, 42(24), 9603-9611.

Nisar, T. M., & Yeung, M. (2018). Twitter as a tool for forecasting stock market movements: A short-window event study. The Journal of Finance and Data Science, 4(2), 101-119.

Nofsinger, J. R. (2005). Social mood and financial economics. The Journal of Behavioral Finance, 6(3), 144-160.

Padmaja, S., & Fatima, S. S. (2013). Opinion mining and sentiment analysis-an assessment of peoples' belief: A survey. International Journal of Ad hoc, Sensor & Ubiquitous Computing, 4(1), 21.

Pagolu, V. S., Reddy, K. N., Panda, G., & Majhi, B. (2016). Sentiment analysis of twitter data for predicting stock market movements. Paper presented at the 2016 international conference on signal processing, communication, power and embedded system (SCOPES).

Peng, Y., & Jiang, H. (2015). Leverage financial news to predict stock price movements using word embeddings and deep neural networks. arXiv preprint arXiv:1506.07220.

Picasso, A., Merello, S., Ma, Y., Oneto, L., & Cambria, E. (2019). Technical analysis and sentiment embeddings for market trend prediction. Expert Systems with Applications, 135, 60-70.

Qian, B., & Rasheed, K. (2007). Stock market prediction with multiple classifiers. Applied Intelligence, 26(1), 25-33.

Regmi, U. R. (2012). Stock market development and economic growth: Empirical evidence from Nepal. Administration and Management Review, 24(1), 1-28.

Ruiz, E. J., Hristidis, V., Castillo, C., Gionis, A., & Jaimes, A. (2012). Correlating financial time series with micro-blogging activity. Paper presented at the Proceedings of the fifth ACM international conference on Web search and data mining.

Sang, C., & Di Pierro, M. (2019). Improving trading technical analysis with tensorflow long short-term memory (LSTM) neural network. The Journal of Finance and Data Science, 5(1), 1-11.

Schumaker, R. P., & Chen, H. (2009). A quantitative stock prediction system based on financial news. Information Processing & Management, 45(5), 571-583.

Sezer, O. B., Ozbayoglu, M., & Dogdu, E. (2017). A deep neural-network based stock trading system based on evolutionary optimized technical analysis parameters. Procedia computer science, 114, 473-480.

Sharma, D., Sabharwal, M., Goyal, V., & Vij, M. (2020). Sentiment analysis techniques for social media data: A review. Paper presented at the First International Conference on Sustainable Technologies for Computational Intelligence.

Shynkevich, Y., McGinnity, T. M., Coleman, S. A., & Belatreche, A. (2016). Forecasting movements of health-care stock prices based on different categories of news articles using multiple kernel learning. Decision support systems, 85, 74-83.

Smith, V. L. (2003). Constructivist and ecological rationality in economics. American economic review, 93(3), 465-508.

Taboada, M., Brooke, J., Tofiloski, M., Voll, K., & Stede, M. (2011). Lexicon-based methods for sentiment analysis. Computational linguistics, 37(2), 267-307.

Taylor, N. (2014). The rise and fall of technical trading rule success. Journal of Banking & Finance, 40, 286-302.

Ticknor, J. L. (2013). A Bayesian regularized artificial neural network for stock market forecasting. Expert Systems with Applications, 40(14), 5501-5506.

Vijh, M., Chandola, D., Tikkiwal, V. A., & Kumar, A. (2020). Stock closing price prediction using machine learning techniques. Procedia computer science, 167, 599-606.

Wang, J.-L., & Chan, S.-H. (2007). Stock market trading rule discovery using pattern recognition and technical analysis. Expert Systems with Applications, 33(2), 304-315.

Wei, L.-Y., Chen, T.-L., & Ho, T.-H. (2011). A hybrid model based on adaptive-network-based fuzzy inference system to forecast Taiwan stock market. Expert Systems with Applications, 38(11), 13625-13631.

Weng, B., Ahmed, M. A., & Megahed, F. M. (2017). Stock market one-day ahead movement prediction using disparate data sources. Expert Systems with Applications, 79, 153-163.

Weng, B., Lu, L., Wang, X., Megahed, F. M., & Martinez, W. (2018). Predicting short-term stock prices using ensemble methods and online data sources. Expert Systems with Applications, 112, 258-273.

Wong, W.-K. (2020). Review on behavioral economics and behavioral finance. Studies in Economics and Finance.

Yu, L. C., Wu, J. L., Chang, P. C., & Chu, H. S. (2013). Using a contextual entropy model to expand emotion words and their intensity for the sentiment classification of stock market news. Knowledge-Based Systems, 41, 89-97.

Zapranis, A., & Tsinaslanidis, P. E. (2012). A novel, rule-based technical pattern identification mechanism: Identifying and evaluating saucers and resistant levels in the US stock market. Expert Systems with Applications, 39(7), 6301-6308.

Zhang, L. (2013). Sentiment analysis on Twitter with stock price and significant keyword correlation.

Zhang, X., Shi, J., Wang, D., & Fang, B. (2018). Exploiting investors social network for stock prediction in China's market. Journal of computational science, 28, 294-303.

Zhu, H., Jiang, Z.-Q., Li, S.-P., & Zhou, W.-X. (2015). Profitability of simple technical trading rules of Chinese stock exchange indexes. Physica A: Statistical Mechanics and its Applications, 439, 75-84.