Implementation of Vision Transformer, a simple way to achieve SOTA in vision classification with only a single transformer encoder, in Pytorch

Orchestrate Swarms of Agents From Any Framework Like OpenAI, Langchain, and Etc for Business Operation Automation. Join our Community: https://discord.gg/DbjBMJTSWD

[CVPR 2024] "CFAT: Unleashing Triangular Windows for Image Super-resolution"

Source code for the GAtt method in "Revisiting Attention Weights as Interpretations of Message-Passing Neural Networks".

Experimental project on building custom LSTM and LSTM with Attention layer for comparison analysis on FTS forecasting (June 2024)

A minimal implementation of a denoising diffusion model in PyTorch.

RWKV is an RNN with transformer-level LLM performance. It can be directly trained like a GPT (parallelizable). So it's combining the best of RNN and transformer - great performance, fast inference, saves VRAM, fast training, "infinite" ctx_len, and free sentence embedding.

Embark on a journey of paralleled/unparalleled computational prowess with FJFormer 🔥 - an arsenal of custom Jax Flax Functions and Utils that elevate your AI endeavors to new heights!

An attention based approach to convert Indian Sign Language to Text using simulated hand gesture data

Researching causal relationships in time series data using Temporal Convolutional Networks (TCNs) combined with attention mechanisms. This approach aims to identify complex temporal interactions. Additionally, we're incorporating uncertainty quantification to enhance the reliability of our causal predictions.

An extension for the code and data of the paper "Human Choice Prediction in Language-based Non-Cooperative Games: Simulation-based Off-Policy Evaluation" (Shapira et al. 2023). This project was conducted by Yogev Namir and Avishag Nevo.

A novel implementation of fusing ViT with Mamba into a fast, agile, and high performance Multi-Modal Model. Powered by Zeta, the simplest AI framework ever.

This repository features a custom-built decoder-only language model (LLM) with a total of 37 million parameters 🔥. I train the model to be able to ask question from a given context

A simple but complete full-attention transformer with a set of promising experimental features from various papers

Xllama🦙 is an Extensible advanced language model framework, inspired by the original Llama model.

PyTorch Implementation of Jamba: "Jamba: A Hybrid Transformer-Mamba Language Model"

QuillGPT is an implementation of the GPT decoder block based on the architecture from Attention is All You Need paper by Vaswani et. al. in PyTorch. Additionally, this repository contains two pre-trained models — Shakespearean GPT and Harpoon GPT, a Streamlit Playground, Containerized FastAPI Microservice, training - inference scripts & notebooks.

Sequence-to-sequence framework with a focus on Neural Machine Translation based on PyTorch

Reference implementation of "Softmax Attention with Constant Cost per Token" (Heinsen, 2024)

Visualizing the attention of vision-language models