Entire Langchain documentation download plus code
Added 2023-08-15 22:44:58 +0000 UTCYou can embed these and create a db.
I have included code files for web crawling and token counting and other tools.
I included documentation for the crawler script
I also included a csv file with all file names and their token counts along with a txt table.
explanation video: https://youtu.be/fGmOiJD65dw
Please check ahead of time if the website allows for crawling and scraping. You can check the robots.txt file in the url for that or their terms and conditions.
Let me know what you think
Code files for AUTO AGI: https://www.patreon.com/posts/code-files-for-87530987
