Build Journal
Enhancements to AEO, Sitemap, and Security Policies — August 24, 2026
I made crucial improvements to AEO, sitemap handling, and security policies in my agentic web project, enhancing performance and reliability.
What shipped
- Expanded AEO Directory — Added /llms-full.txt for comprehensive portal links.
- Improved Sitemap Handling — Stopped incorrect lastmod timestamps on static routes.
- Enhanced Security Policies — Implemented nonce-less CSP and security.txt compliance.
- Real 404 Error Pages — Introduced genuine 404 pages to improve SEO.
- Authentication Fix — Updated auth API to reduce console errors for signed-out users.
Today, I set out to enhance the performance, security, and SEO aspects of my agentic web project, focusing on AEO improvements, sitemap optimization, and implementing robust security policies. I shipped ten feature and improvement commits along with one fix during a solid five-hour development session.
The most significant addition was the expanded `/llms-full.txt`, which now serves as a comprehensive public portal directory. With this enhancement, agents can access a complete matrix of per-portal links along with their respective costs and industry URLs in a single fetch. This change is crucial as agents increasingly probe the `/llms-full.txt` first. The integration of this feature required me to think through the data structure and ensure it met the needs of the agents without introducing unnecessary complexity.
I also tackled issues in the security content security policy (CSP) by allowing specific trusted sources, including `static.cloudflareinsights.com` and the Jitsi host, to ensure that the necessary scripts load without issues. This change was prompted by the first-enforce-day console reports that showed blocked scripts, which would have otherwise hindered functionality. The debugging process involved analyzing the CSP violations and refining the allowlist, which took longer than expected, but ultimately resulted in a more secure environment.
On the SEO front, I made a critical adjustment by stopping the practice of stamping `lastmod=now` on static routes. This was causing Google to distrust the last modified field, as the hourly regeneration was misleading. Instead, I’ve configured evergreen routes to omit `lastModified` when it’s not genuinely updated. This decision was grounded in the necessity for Google to accurately index my content, and I’m hopeful this will improve the overall SEO performance of my site.
Another notable enhancement was the implementation of a real 404 page with a genuine 404 status. Previously, the redirect-to-map catch-all was transforming every dead or mistyped URL into a soft-404, which cluttered up Google Search Console with noise. Now, users will encounter a proper 404 page, while crawlers will receive an honest status, which should help with both user experience and search engine indexing.
I also focused on tightening up security measures across the board. I added a `/well-known/security.txt` file in compliance with RFC 9116, which provides crucial contact information and security disclosures. This addition is part of my broader strategy to enhance transparency and trustworthiness in my platform. Furthermore, I enforced a full nonce-less baseline policy in the CSP to eliminate potential vulnerabilities. This involved careful consideration of how cached HTML interacts with security policies, and it took a fair amount of time to ensure that the implementation was both effective and efficient.
The fix I implemented for the authentication API endpoint was another highlight of the day. The previous response structure was causing confusion for users and errors in the console for every signed-out visitor. By ensuring that the `/api/auth/me` endpoint returns a `200` status with `{user:null}` instead of a `401`, I’ve streamlined the user experience and reduced unnecessary error noise. It’s small changes like these that, when aggregated, contribute significantly to the overall quality of the user experience.
Looking back at today’s work, I can see how each of these enhancements aligns with my vision of building a one-man-show company with a billion-dollar valuation. By leveraging AI assistance and focusing on critical areas like AEO, sitemap optimization, and security, I’m laying a solid foundation for the future. This is a grind, and while it’s tempting to rush into new features, it’s the fundamentals that will sustain growth and trust in the long run. As I continue to build in public, I’m excited about the improvements I’m making, both in terms of technology and user experience, and I’m eager to see how these changes will impact my project moving forward.