RTX 3090 · 12h+ Fully Local Autoresearch

Autoresearch Loop Results: qwen3.5:9b-64k on RTX 3090

Key Findings

After running a 12-hour autoresearch loop with the RTX 3090, the optimal configuration for maximizing output tokens per second (output_tokens_per_second) was identified. The best performance achieved was 100.18 tokens/s, with the following parameters:

• Context length: 4096 tokens
• Batch size: 512

Performance Trends

The initial benchmark started at 99.45 tokens/s and showed incremental improvements with adjustments to context length and batch size. The peak performance was observed at 4096 tokens context length and 512 batch size.

Analysis

1. Context Length Impact: Increasing context length beyond 4096 tokens did not yield better performance, indicating that the model's optimal context window is likely around 4096 tokens.
2. Batch Size Optimization: A batch size of 512 provided the best balance between throughput and resource utilization. Larger batch sizes did not maintain the same level of performance, suggesting potential bottlenecks in GPU memory or computational efficiency.
3. GPU Residency: The research confirmed that maintaining full GPU residency is crucial for achieving peak performance. The optimal configuration ensures that the GPU is fully utilized without excessive memory contention.

Recommendations

• Focus on Balance: Users should prioritize balancing context length and batch size rather than pursuing larger contexts alone. The optimal configuration is a trade-off between these parameters.
• Monitor Resource Usage: Keep an eye on GPU memory and computational resources to avoid bottlenecks. The RTX 3090's performance is highly dependent on efficient resource management.
• Iterative Testing: Further experiments with different context lengths and batch sizes can help refine the optimal configuration for specific use cases.

This research provides a clear pathway for maximizing output tokens per second on the RTX 3090, ensuring efficient and effective model deployment.