tilTiltanymTiltanym2411-60762709-135XИнститут языкознания имени А.Байтурсынулы КН МОН РК10.55491/2411-6076-2026-1-135-149til-2118Research ArticleПРИКЛАДНОЕ ЯЗЫКОЗНАНИЕAPPLIED LINGUISTICSСоздание больших языковых моделей в казахском языке: казахстанский опыт и глобальные трендыBuilding Large Language Models in Kazakh Language: Kazakhstan’s Experience and Global TrendsЖанабековаА. А.ZhanabekovA.

Айман Абдильдаевна Жанабекова, доктор филологических наук, ассоциированный профессор

 

 

Aiman Zhanabekova, Doctor of Philological Sciences

Almaty

aiman_miras@mail.ruhttps://orcid.org/0009-0003-9679-5572ҮсеновМ.UsenovM.

Мирас Үсенов, 

Miras Usenov,Student 

Almaty

ussenov.miras08@gmail.comhttps://orcid.org/0000-0001-9842-3324ТлегеноваГ. Б.TlegenovaG.

Гульден Бакытказыевна Тлегенова, доктор философии (PhD)

 

Gulden Tlegenova, Doctor of Philosophy (PhD)

Almaty

 

gulden_20.88@list.ruИнститут языкознания имени Ахмета БайтурсынулыКазахстанAkhmet Baitursynuly Institute of LinguisticsKazakhstanНархоз университетіКазахстанNarxoz UniversityKazakhstan20261504202601135149Copyright © Жанабекова А.А., Үсенов М., Тлегенова Г.Б., 20262026Жанабекова А.А., Үсенов М., Тлегенова Г.Б.Zhanabekov A., Usenov M., Tlegenova G.Данная работа распространяется под лицензией Creative Commons Attribution 4.0.This work is licensed under a Creative Commons Attribution 4.0 License.https://www.tiltanym.kz/jour/article/view/2118

В статье анализируются научно-практические аспекты создания Большой языковой модели (LLM) на основе искусственного интеллекта, ориентированной на казахский язык в качестве базовой платформы на 2024-2026 годы. Актуальность исследования обусловлена задачами формирования полноценной цифровой репрезентации казахского языка, сохранения национальной идентичности и языкового суверенитета в условиях цифровизации. Цель работы – описать теоретические принципы LLM, структуру нейросетевой модели трансформерного типа, состав и разметку датасета, а также показать их соответствие национальным и международным этико-правовым нормам. Корпус данных охватывает национальный менталитет, историкокультурные традиции и действующее законодательство. Материалы, сгруппированные по нескольким ключевым категориям, включают связки «вопрос – ответ», экспертные комментарии и формулируются с учётом этических требований. Рассматриваются современные методы обработки текста, подходы к снижению числа ошибочных и вымышленных ответов модели, а также качественная оценка полученных результатов. Научная новизна состоит в системном описании первой комплексной LLM для казахского языка; практическая значимость – в возможности её применения в образовании, научной аналитике, цифровых сервисах и при реализации государственной языковой политики.  

The article analyzes the scientific and practical aspects of developing an artificial intelligence-based Large Language Model (LLM) oriented toward the Kazakh language as a basic platform for 2024-2026. The relevance of the study is determined by the need to create a full-fledged digital representation of Kazakh, preserve national identity, and ensure linguistic sovereignty in the context of digitalization. The aim of the work is to describe the theoretical principles of the LLM, the structure of a transformer-type neural network, and the composition and annotation of the dataset, as well as to demonstrate their compliance with national and international ethical and legal norms. The data corpus covers national mentality, historical and cultural traditions, and current legislation. The materials, grouped in several key categories, include question–answer pairs and expert commentary and are formulated with ethical requirements in mind. The paper discusses modern methods of text processing, approaches to reducing erroneous and fabricated model outputs, and a qualitative assessment of the results obtained. The scientific novelty is reflected in the systematic description of the first comprehensive large language model (LLM) for the Kazakh language; its practical significance is demonstrated by its potential applications in education, scientific analytics, digital services, and the implementation of state language policy. 

искусственный интеллектLLMKazLLMязыковая модельChatGPTкорпусartificial intelligenceLLMKazLLMlanguage modelChatGPTcorpusСтатья подготовлена в рамках проекта программно-целевого финансирования Комитета науки Министерства науки и высшего образования Республики Казахстан ИРН №BR24993001 «Создание большой языковой модели (LLM) для поддержки казахского языка и технологического прогресса».The article was prepared within the framework of the program-targeted funding project of the Committee of Science of the Ministry of Science and Higher Education of the Republic of Kazakhstan IRN BR24993001 “Development of a Large Language Model (LLM) to support the Kazakh language and technological progress.”ReferencesБектаев Қ.Б., Жұбанов А.Қ., Мырзабеков С., Белботаев А.Б. М.О. Әуезовтің 20 томдық шығармалар текстерінің жиілік сөздіктері. – Алматы-Түркістан, 1995. – 346 б.Bektaev, Q.B., Zhubanov, A.Q., Myrzabekov, S., Belbotaev, A.B. (1995) M.O. Auezovting 20 tomdyq shygarmalar teksterining zhiilik sozdikteri. Almaty-Turkistan. 346 b. [Bektayev, K.B., Zhubanov, A.K., Myrzabekov, S., Belbotayev, A.B. (1995) Frequency dictionaries of texts of 20-volume works of M.O. Auezov. Almaty-Turkestan. 346 p.] (in Kazakh)Жұбанов А., Жаңабекова А., Карбозова Б., Қожахметова А. (2016) Қазақ тілінің жиілік сөздігі. – Алматы: Қазақ тілі, 2016. – 792 б.Bender, E.M., Gebru, T., McMillan-Major, A., Shmitchell, S. (2021) On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? FAccT. https://dl.acm.org/doi/10.1145/3442188.3445922 (in English)Токаев К.-Ж. Послание Главы государства народу Казахстана «Казахстан в эпоху искусственного интеллекта: актуальные задачи и их решения через цифровую трансформацию». – Астана, 8 сентября 2025 года.BigScience Workshop et al. (2022) Bloom: A 176B-Parameter Open-Access Multilingual Language Model. arXiv. https://arxiv.org/abs/2211.05100 (in English)Bender, E.M., Gebru, T., McMillan-Major, A., Shmitchell, S. On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? FAccT, 2021. https://dl.acm.org/doi/10.1145/3442188.3445922Bommasani, R., et al. (2021) On the Opportunities and Risks of Foundation Models. arXiv preprint. https://arxiv.org/abs/2108.07258 (in English)BigScience Workshop et al. Bloom: A 176B-Parameter Open-Access Multilingual Language Model. arXiv, 2022. https://arxiv.org/abs/2211.05100Brown, T.B., Mann, B., Ryder, N., et al. (2020) Language Models are Few-Shot Learners. NeurIPS. https://arxiv.org/abs/2005.14165 (in English)Bommasani, R., et al. On the Opportunities and Risks of Foundation Models. 2021. arXiv preprint. https://arxiv.org/abs/2108.07258Bubeck, S., et al. (2023) Sparks of Artificial General Intelligence: Early experiments with GPT-4. https://arxiv.org/abs/2303.12712 (in English)Brown, T.B., Mann, B., Ryder, N., et al. Language Models are Few-Shot Learners. 2020. NeurIPS. https://arxiv.org/abs/2005.14165Chung, H.W., Hou, L., et al. (2022) Scaling Instruction-Finetuned Language Models. arXiv. Conneau, A., Khandelwal, K. et al. (2020) Unsupervised cross-lingual representation learning at scale. NeurIPS. (in English)Bubeck, S., et al. Sparks of Artificial General Intelligence: Early experiments with GPT-4. 2023. https://arxiv.org/abs/2303.12712Goodfellow, I., Bengio, Y., Courville, A. (2016) Deep Learning. MIT Press. P. 801. (in English)Chung, H.W., Hou, L., et al. Scaling Instruction-Finetuned Language Models. arXiv, 2022.Google Kazakh NLP team (2021) Kazakh Language and Speech Resources for the Next Generation. https://sites.google.com/view/kaznlp (in English)Conneau, A., Khandelwal, K. et al. Unsupervised cross-lingual representation learning at scale. NeurIPS, 2020.Hoffmann, J. et al. (2022) Training Compute-Optimal Large Language Models. https://arxiv.org/abs/2203.15556 (in English)Goodfellow, I., Bengio, Y., Courville, A. Deep Learning. MIT Press, 2016. P. 801.Khan, A., Ismail, S.A. (2022) Challenges for Building Natural Language Generation Models in Turkic Languages. IEEE Access. (in English)Google Kazakh NLP team. Kazakh Language and Speech Resources for the Next Generation. 2021. https://sites.google.com/view/kaznlpLevine, Y., et al. (2021) The Depth-to-Width Ratio for ReLU Networks: Past and Present. NeurIPS. (in English)Hoffmann, J. et al. Training Compute-Optimal Large Language Models. 2022. https://arxiv.org/abs/2203.15556Serikbayeva, A., Olach, Z. (2025) Kazakh language in the context of globalization and digital transformation: language changes. Tiltanym. No.1 (97). Pp. 151-160. (in English)Khan, A., Ismail, S.A. Challenges for Building Natural Language Generation Models in Turkic Languages. IEEE Access, 2022.Tokaev, K.-Zh. (2025) Poslanie glavy gosudarstva narodu Kazahstana «Kazahstan v jepohu iskusstvennogo intellekta: aktual'nye zadachi i ih reshenija cherez cifrovuju transformaciju». Astana, 8 sentjabrja 2025 goda. [Tokayev, K.-J. (2025) Address of the Head of State to the People of Kazakhstan “Kazakhstan in the Era of Artificial Intelligence: Current Tasks and Their Solutions through Digital Transformation.” Astana, September 8, 2025.] (in Russian)Levine, Y., et al. The Depth-to-Width Ratio for ReLU Networks: Past and Present. 2021. NeurIPS. https://arxiv.org/abs/2202.03841Tulegenov, E. Myrzabekov, S. Abduvaliyev, A. (2023) Kazakh LLM: Technological Foundation and Challenges in Local Adaptation. IMBISS Project Report, Almaty, Kazakhstan. (in English)Serikbayeva, A., Olach, Z. Kazakh language in the context of globalization and digital transformation: language changes. Tiltanym, 2025. No.1 (97). Pp. 151-160. (in English)Vaswani, A., et al. (2017) Attention is All You Need. NeurIPS. https://arxiv.org/abs/1706.03762 (in English)Tulegenov, E. Myrzabekov, S. Abduvaliyev, A. Kazakh LLM: Technological Foundation and Challenges in Local Adaptation. IMBISS Project Report. 2023. Almaty, Kazakhstan.Weidinger, L., et al. (2022) Ethical and social risks of harm from language models. Science, 376(6595). Pp. 939-944. (in English)Vaswani, A., et al. Attention is All You Need. NeurIPS. 2017. https://arxiv.org/abs/1706.03762Wilkinson, S., Cunningham, S.J. (2022) Low-Resource Language Modelling and Data Augmentation». Machine Learning for NLP. https://arxiv.org/abs/2310.15799 (in English)Weidinger, L., et al. Ethical and social risks of harm from language models. Science, 376(6595), 939-944, 2022.Zhang, S., Sun, A. et al. (2021) Ethics and Bias in Large Language Models: A Survey. ACM Computing Surveys. (in English)Wilkinson, S., Cunningham, S.J. Low-Resource Language Modelling and Data Augmentation. Machine Learning for NLP, 2022. https://arxiv.org/abs/2310.15799Zhubanov, A., Zhanabekova, A., Karbozova, B., Qozhahmetova, A. (2016) Qazaq tilining zhiilik sozdigi. Almaty: Qazaq tili. 792 b. [Zhubanov, A., Zhanabekova, A., Karbozova, B., Kozhakhmetova, A. (2016) Frequency dictionary of the Kazakh language. Almaty: Qazaq tili. 792 p.] (in Kazakh)Zhang, S., Sun, A. et al. Ethics and Bias in Large Language Models: A Survey. ACM Computing Surveys, 2021.Zhang, S., Sun, A. et al. Ethics and Bias in Large Language Models: A Survey. ACM Computing Surveys, 2021.

The authors declare that there are no conflicts of interest present.