Build Journal
Agentic Web AEO Enhancements & Security Fixes — August 24, 2026
I made significant enhancements to AEO, security, and SEO on the agentic web, addressing critical bugs and optimizing performance for better user experience.
What shipped
- Expanded LLM Directory — Launched /llms-full.txt for complete public portal access.
- CSP Policy Enhancements — Implemented nonce-less baseline policy for improved security.
- Real 404 Page — Configured genuine 404 responses for better user experience.
- SEO Sitemap Optimization — Removed lastmod=now from static routes for accurate indexing.
- Auth Error Fix — Adjusted /api/auth/me endpoint to prevent console errors.
Today, I focused on enhancing the agentic web's AEO capabilities while tightening security measures and improving SEO. I shipped a total of 11 commits over a solid 5-hour session, and I’m proud to report significant progress across multiple areas, including dependencies, security policies, sitemap updates, and error handling.
One of the standout features was the addition of the /llms-full.txt endpoint. This expanded resource serves as a complete public portal directory, linking to various LLMs and providing a comprehensive cost, metro, and industry URL matrix. By linking it from /llms.txt, I ensured that agents probing for this information would access the most complete version first. This enhancement is vital as it aligns with the growing trend of agents searching for richer data in one go, making the agentic web more robust and user-friendly.
I also tackled some pressing security issues today. A major fix involved enforcing a nonce-less baseline policy for Content Security Policy (CSP). This change was necessary to eliminate any potential vulnerabilities that could arise from cached or ISR HTML becoming stale. I tightened the script, connect, and frame allowlists while ensuring that object-src was set to none. This effort was essential because it helps maintain a more secure environment and fosters trust in the platform. I also removed some outdated allowances, such as http://ip-api.com and unsafe-eval, which had been flagged in weeks of reports.
On the SEO front, I made a critical adjustment to how I manage static routes in the sitemap. Previously, I was stamping lastmod=now on static routes, which led Google to distrust the field where it should be reliable. With the new approach, evergreen routes will omit lastModified dates honestly, ensuring that Google gets accurate information about when content was last updated. This change reflects my commitment to providing a transparent and trustworthy experience for users and crawlers alike.
Another significant improvement was the implementation of a real 404 page with an authentic 404 status. The previous setup redirected users to a map page, which not only created confusion for users but also cluttered Google Search Console with soft-404 reports. Now, when users hit a dead or mistyped URL, they receive a genuine 404 response, guiding them back to the main site without dead ends. This fix was a long time coming and will undoubtedly enhance user experience and SEO performance.
I also fixed a bug in the authentication process where the /api/auth/me endpoint was returning a 200 status with {user:null} for anonymous users instead of a 401. This miscommunication led to red console errors on every page load for signed-out visitors. I adjusted the response to treat user:null as a definitive signed-out signal, while also preserving the 401/403 branches for revoked sessions. This change not only improved the user experience but also alleviated unnecessary noise in the console logs.
Throughout the day, I utilized Claude Code in VS Code to assist with various tasks, and I’m continually amazed at how much more efficient I can be with AI as my partner. While building this one-man-show towards a billion-dollar valuation, I find that leveraging AI tools allows me to focus on strategic enhancements rather than getting bogged down in repetitive coding tasks.
Reflecting on the challenges I faced today, I encountered some headwinds, particularly with the CSP policies and the 404 handling. Diagnosing the issues with the CSP required digging through console logs and understanding how different scripts were being blocked. Similarly, the 404 fix involved analyzing user behavior to ensure that users were directed appropriately without hitting dead ends. These challenges, while time-consuming, ultimately lead to a more secure and user-friendly platform.
As I close out the day, I feel accomplished with the enhancements made to the agentic web. Each improvement not only pushes the platform forward but also reinforces my vision of creating a seamless, intelligent, and responsive environment for users. With each commit, I’m building towards a future where the agentic web is not just functional, but exceptional.