体育赛事投注记录

advertisement

A New Methodology for Language Identification in Social Media Code-Mixed Text

  • Yogesh GuptaEmail author
  • Ghanshyam Raghuwanshi
  • Aprna Tripathi
Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 1141)

Abstract

nowadays, transliteration is one of the hot research areas in the field of natural language processing. transliteration means that transferring a word from one language to another language and it is mostly used in cross-language platforms. generally, people use code-mixed language for sharing their views on social media like twitter, whatsapp, etc. code-mixed language means one language is written using another language script and it is very important to identify the languages used in each word to process such type of text. therefore, a deep learning model is implemented using bidirectional long short-term memory (blstm) for indian social media texts in this paper. this model identifies the origin of the word from language perspective in the sequence based on the specific words that have come before it in the sequence. the proposed model gives better accuracy for word-embedding model as compared to character embedding.

Keywords

Natural Language Processing Character embedding Word embedding Machine learning 

References

  1. 1.
    Kim, Y.: Convolutional neural networks for sentence classification. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP), Doha, Qatar, pp. 1746–1751 (2014)
  2. 2.
    King, B., Abney, S.: Labeling the languages of words in mixed-language documents using weakly supervised methods. In: Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 1110–1119 (2013)
  3. 3.
    Nguyen, D., Dogruoz, A.S.: Word level language identification in online multilingual communication. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing, pp. 857–862 (2013)
  4. 4.
    Das, A., Gamback, B.: Identifying languages at the word level in code-mixed Indian social media text. In: Proceedings of the 11th International Conference on Natural Language Processing, India, pp. 378–387(2014)
  5. 5.
    Sequiera, R., Choudhury, M., Gupta, P., Rosso, P., Kumar, S., Banerjee, S., Naskar, S., Bandyopadhyay, S., Chittaranjan, G., Das, A., Chakma, K.: Overview of FIRE-2015 shared task on mixed script information retrieval, vol. 1587, pp. 19–25 (2015)
  6. 6.
    Jhamtani, H., Bhogi, S.K., Raychoudhury, V.: Word-level language identification in bi-lingual code-switched texts. In: Proceedings of 28th Pacific Asia Conference on Language, Information and Computation, pp. 348–357 (2014)
  7. 7.
    Ethiraj, R., Shanmugam, S., Srinivasa, G., Sinha, N.: NELIS—named entity and language identification system: shared task system description. FIRE Workshop 1587, 43–46 (2015)
  8. 8.
    Bhargava, R., Sharma, Y., Sharma, S.: Sentiment analysis for mixed script indic sentences. In: Proceedings of International Conference on Advances in Computing, Communications and Informatics, ICACCI, India, pp. 524–529 (2016)
  9. 9.
    Sharma, S., Srinivas, P., Balabantaray, R.: Emotion detection using online machine learning method and TLBO on mixed script. In: Language Resources and Evaluation Conference, vol. 10, no. 5, pp. 47–51 (2016)
  10. 10.
    Bali, K., Jatin, S., Choudhury, M., Vyas, Y.: I am borrowing ya mixing? An analysis of English-Hindi code mixing in Facebook. In: Proceedings of The First Workshop on Computational Approaches to Code Switching, EMNLP, pp. 116–126 (2014)
  11. 11.
    Rao, P.R.K., Devi, S.L.: CMEE-IL: code mix entity extraction in indian languages from social media Text@FIRE 2016—an overview. FIRE Workshop 1737, 289–295 (2016)
  12. 12.
    Sapkal, K., Shrawankar, U.: Transliteration of secured SMS to Indian regional language. Proc. Comput. Sci. 78, 748–755 (2016)
  13. 13.
    Zubiaga, A., Vicente, I.S., Gamallo, P., Pichel, J.R., Alegria, I., Aranberri, N., Ezeiza, A., Fresno, V.: TweetLID: a benchmark for tweet language identification. Lang. Resour. Eval. 50(4), 729–766 (2015)
  14. 14.
    Alekseev, A., Nikolenko, S.: Word embedding for user profiling in online social networks. Computacion y Sistemas 21(2), 203–226 (2017)
  15. 15.
    Chaudhary, J., Patel, A.C.: Bilingual machine translation using RNN based deep learning. Int. J. Sci. Res. Sci. Eng. Technol. 4(4), 1480–1484 (2018)
  16. 16.
    Samuel, K.C.: Exploring language; some specificities, complexities and limitations in human communication and social interaction in multi-cultural contexts. Adv. J. Soc. Sci. 5(1), 26–36 (2019)
  17. 17.
    Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)
  18. 18.
    Mikolov, T., Sutskever, I., Chen, K., Corrado, G., Dean, J.: Distributed representations of words and phrases and their compositionality. In: Proceedings of the 26th International Conference on Neural Information Processing System, vol. 2, pp. 3111–3119 (2013)
  19. 19.
    Jamatia, A., Das, A.: Task report: tool contest on POS tagging for code-mixed Indian social media (Facebook, Twitter, and WhatsApp) text. In: Proceedings of ICON (2016)
  20. 20.
    Banerjee, S., Chakma, K., Naskar, S., Das, A., Rosso, P., Bandyopadhyay, S., Choudhury, M.: Overview of the mixed script information retrieval (MSIR). In: CEUR Workshop Proceedings, vol. 1737, pp. 94–99 (2016)

Copyright information

© Springer Nature Singapore Pte Ltd. 2021

Authors and Affiliations

  • Yogesh Gupta
    • 1
    Email author
  • Ghanshyam Raghuwanshi
    • 2
  • Aprna Tripathi
    • 3
  1. 1.Department of Computer Science and EngineeringManipal University JaipurJaipurIndia
  2. 2.Department of Computer and Communication EngineeringManipal University JaipurJaipurIndia
  3. 3.Department of Computer Engineering and ApplicationsGLA UniversityMathuraIndia

Personalised recommendations