A developer has released a modified version of the Qwen3-4B-Instruct model, dubbed 'qwen3-4b-deslop,' specifically engineered to strip away the stylistic hallmarks often associated with large language models (LLMs). These 'tics' include the frequent use of em dashes, repetitive list structures of three, and specific transition phrases like 'delve' or 'seamless.' By using a combination of distillation and neuron editing, the developer aimed to produce a model that generates text closer to human writing styles without sacrificing its ability to provide correct factual information.
This image represents the project's goal of removing 'slop' from the Qwen3-4B-Instruct model's output.
The training process involved multiple rounds of distillation. The developer first identified specific 'assistant' behaviors within the model's weights and used a 'teacher' model to generate high-quality responses that passed a series of style and task-fit checks. One notable technical step involved a 'neuron edit,' where the sign of four specific MLP (multi-layer perceptron) neurons was flipped to remove specific linguistic habits like the 'not X, but Y' construction. In the final iteration, the model was trained on 161 high-quality answers that successfully avoided 'slop'—a metric the developer created to weight and count these stylistic habits. The result was a significant reduction in the 'slop score' from 121 to 12.5 per 1000 words, with many outputs scoring within the range of human-written prose.
Despite the reduction in stylistic artifacts, the model faces several limitations. The developer noted that the resulting prose can be 'flat' or overly declarative, and the model still occasionally hallucinates names and places to provide concrete details. Furthermore, while the model performs well on short English prose, its performance on code, long-form conversations, or other languages remains unverified. The developer plans to expand this methodology to larger models and eventually use a base model—one that has not undergone 'assistant' training—to see if a more nuanced 'taste' can be instilled into the weights without the flat tone of the current iteration.
Technical details for the 'deslop' model include a LoRA adapter (126 MB) and a 175-byte list of weight columns for the neuron edit. The model is available in a quantized GGUF format for use with tools like Ollama or llama.cpp, as well as full-precision weights for use with the Hugging Face transformers library. The developer emphasizes that the current results are based on a self-authored style guide and have not yet undergone a blind human test to determine if the 'de-slopped' text is genuinely preferred over the original."