When Time Becomes Your Enemy

The Problem

Users consistently received answers based on outdated PTO policies. The current policy existed in the knowledge base but was buried in a different section with poor metadata. 40% of policy queries returned outdated information, creating confusion and potential compliance issues.

Diagnosis

Solution: Multi-layered temporal awareness system

# Temporal metadata enrichment
docs = [
    Document(
        text=policy_text,
        metadata={
            "doc_id": "hr_policy_v3",
            "version": "3.2.1",
            "effective_date": "2024-01-01",
            "expiry_date": "2024-12-31",
            "supersedes": ["hr_policy_v2", "hr_policy_v1"],
            "section": "pto_policy"
        }
    )
]

# Filter by effective date
current_date = datetime.now().isoformat()
retriever = index.as_retriever(
    filters={
        "effective_date": {"$lte": current_date},
        "expiry_date": {"$gte": current_date}
    },
    similarity_top_k=5
)

class TemporalReranker:
    def rerank(self, nodes, query):
        for node in nodes:
            days_old = (datetime.now() - node.metadata["last_updated"]).days
            temporal_boost = max(0.1, 1.0 - (days_old / 365))
            node.score *= temporal_boost
        return sorted(nodes, key=lambda x: x.score, reverse=True)

Results

Key Lessons

• Temporal metadata is essential for any time-sensitive information
• Hard filters prevent outdated content from reaching users
• Automated monitoring catches temporal regressions before users do
• Version control for knowledge bases is as important as code version control