Task Queues: Enhanced Reliability for AI Features

 Feature Name:Task Queues

 Product Area:AI Infrastructure (SDC and other AI features)

 Impact of Change: Reliability Improvement

 Roles Impacted: All users of SpotDraft AI features

Description:

We've implemented a robust new queueing system to significantly improve the reliability of our AI features, particularly Smart Document Compare (SDC)!

Problem Addressed:

Previously, during high-volume periods, our system would occasionally hit LLM (Large Language Model) quota limits from our third-party providers like Google Gemini and OpenAI. When this happened:

• Some requests would get stuck and never get completed
• Key point extractions would fall through the cracks
• Users would see a "stuck" state with no progress
• If one customer was making many requests, it could affect other customers trying to use the system simultaneously

Solution Overview:

Our new Task Queues system creates a persistent, ordered queue that intelligently manages sudden spikes in traffic: Requests are properly queued and tracked, and Automatic retry mechanisms trigger when third-party services throttle us

Important callouts:

Possible slight delays: During extremely high usage periods, users might notice SDC processing taking longer than usual. This is expected and means the requests are being managed and will be completed.

Increased completion rate: While individual requests might occasionally take longer, the overall reliability and completion rate of AI features are significantly higher.

No action needed: This is a behind-the-scenes infrastructure improvement that requires no changes to how you use SpotDraft.

How to Enable: This feature is being rolled out gradually, reach out to support for enablement.