BroAlanTaps
/

Stage1-PCC-Lite-4x

Feature Extraction

text-generation

text-generation-inference

Model card Files Files and versions

Model Card for Model ID

PCC-Lite-4x Model:

Compressor: Continue pre-training based on GPT2-Large

Downloads last month: 7

Safetensors

Model size

0.8B params

Tensor type

BF16

·

Model tree for BroAlanTaps/Stage1-PCC-Lite-4x

Base model

openai-community/gpt2-large

Finetuned

(127)

this model

Dataset used to train BroAlanTaps/Stage1-PCC-Lite-4x

Collection including BroAlanTaps/Stage1-PCC-Lite-4x

PCC-Pretrained

Pretraining Context Compressor for Large Language Models with Embedding-Based Memory • 15 items • Updated Sep 22 • 1