Maybe it’s time to just download it and Wikipedia, then point an LLM / vectordb at it.
Perhaps the way to solve for X is having a scraper serve the data via torrent and/or IPFS vs centralized systems like nitter and xcancel.
Maybe it’s time to just download it and Wikipedia, then point an LLM / vectordb at it.