pax_global_header00006660000000000000000000000064145432570110014514gustar00rootroot0000000000000052 comment=7d97cbd7069ac9820d47496bceac1169c61e9e16 rwkv-v5-12b-one-state-chat-16k/000077500000000000000000000000001454325701100161745ustar00rootroot00000000000000rwkv-v5-12b-one-state-chat-16k/README.md000066400000000000000000000005001454325701100174460ustar00rootroot00000000000000# RWKV-v5-12B-one-state-chat-16k ####forked from https://huggingface.co/xiaol/RWKV-v5-12B-one-state-chat-16k [README from original repo](https://openi.pcl.ac.cn/StarrySky/RWKV-v5-12B-one-state-chat-16k/src/branch/master/README_from_original_repo.md) ### The files of this repo are stored in the "models" and "datasets"rwkv-v5-12b-one-state-chat-16k/README_from_original_repo.md000066400000000000000000000031141454325701100234060ustar00rootroot00000000000000--- license: apache-2.0 datasets: - PygmalionAI/PIPPA - Norquinal/claude_multiround_chat_30k - garage-bAInd/Open-Platypus - kaist-ai/CoT-Collection - BAAI/COIG-PC - togethercomputer/Long-Data-Collections - tasksource/icl-symbol-tuning-instruct --- # Release date: December 18th finetuned from the state-of-the-art (SOTA) model, RWKV v5 12B one state base! More details will be provided soon. Enjoy the incredible performance of this model, which is optimized for systems with 24GB of VRAM and supports fp16. It can be fine-tuned using a single A100 GPU. To execute this model, utilize the [RWKV Runner](https://github.com/josStorer/RWKV-Runner) tool. # Finetuned from [Mobius 12B base](https://huggingface.co/xiaol/Mobius-12B-base) # Usage - [RWKV next web](https://rwkv-next-web.ai-creator.net/) - if use with [RWKV runner](https://github.com/josStorer/RWKV-Runner) or [ai00 server](https://github.com/cgisky1980/ai00_rwkv_server), change default vocabs(tokenizer) by [this one](https://huggingface.co/xiaol/RWKV-v5-12B-one-state-chat-16k/blob/main/rwkv_vocab_v20230424.txt) # Important Notes After overfitting certain instructions and weakening others, it is necessary to use completion or simulate dialogues. - **completion prompt** = 'User: make this content longer:\nxxxxxx\n\nAssistant: ok, longer content is' # Data format `User:xxxx\n\nAssistant:xxx\n\nUser:xxxx\n\nAssistant:xxx\n\n` If you desire optimal performance to run this model,utilize this format and these [vocabs](https://huggingface.co/xiaol/RWKV-v5-12B-one-state-chat-16k/blob/main/rwkv_vocab_v20230424_train.txt)