causal-language-modeling

Repository for My HuggingFace Natural Language Processing Projects

Auto-regressive causal language model for molecule (SMILES) and reaction template (SMARTS) generation based on the Hugging Face implementation of OpenAI's GPT-2 transformer decoder model

Codebase for Linguistic Collapse: Neural Collapse in (Large) Language Models [NeurIPS 2024] [arXiv:2405.17767]

Transformers Intuition

Links to my repositories, where I implement a wide variety of Natural Language Processing models using TensorFlow and Hugging Face.

A multi-threaded GitHub scraper to collect Python code with docstrings from public repositories, creating a well-documented dataset for the JaraConverse LLM model.

Open source implementation of AMALIA: A Fully Open Large Language Model for European Portuguese

This repository is for the paper Lexical Substitution as Causal Language Modeling. In Proceedings of the 13th Joint Conference on Lexical and Computational Semantics (*SEM 2024), Mexico City, Mexico. Association for Computational Linguistics.

CUDA-native residual-frame Delta memory for causal language models: full-matrix prefix geometry, rank-one recurrent updates, exact local similarity, Hugging Face integration, and CUDA Graph training.

Change the Bert model to a GPT-style autoregressive decoder.

HelixLM - a recurrent heterogeneous graph neural LLM combining biological neural column random wiring, hybrid attention, and Mamba-2 for hyperpersonalized and on-device AI.

Causal language modeling and intent classification using GPT-2.

A quick and easy way to interact with open-source LLMs.

Fine-tuning (or training from scratch) the library models for language modeling on a text dataset for GPT, GPT-2, ALBERT, BERT, DitilBERT, RoBERTa, XLNet... GPT and GPT-2 are trained or fine-tuned using a causal language modeling (CLM) loss while ALBERT, BERT, DistilBERT and RoBERTa are trained or fine-tuned using a masked language modeling (MLM…

An AI generated picturebook.

Course materials for the Machine Learning for NLP course taught by Sameer Singh for the Cognitive Science summer school 2022.

Rescoring Automatic Speech Recognition using Large Language Models

Text-to-SQL generation with BART and GPT-2, including fine-tuning, SQL normalization, exact-match evaluation, and error analysis.