Dify #38518: fix orphaned data on KB delete

Job description

Source issue: https://github.com/langgenius/dify/issues/38518 Repo: https://github.com/langgenius/dify Task: Fix or propose a robust solution for the dataset deletion path that can leave orphaned segments, child chunks, documents, and pgvector tables when a knowledge base is deleted during active indexing. Deliverables: Review the issue and the cleanup/indexing code paths involved. Implement a durable cleanup fix and/or deletion guard that addresses the reported race and rollback behavior. Include tests, migration notes, or a validation plan for large datasets. Share the exact files changed and any caveats for self hosted operators. Acceptance criteria: The solution directly addresses issue 38518. Orphaned data/vector table behavior is prevented or reliably cleaned up. The approach is practical for large datasets and failure scenarios. Include links or commit references for the work product. Please mention issue 38518 in your application and summarize your proposed implementation plan.