BuilderIO/gpt-crawler
gpt-crawler
Crawl a site to generate knowledge files to create your own custom GPT from a URL
Builder
BuilderIO
BuilderIO • individual
Stars
22,222
Using upstream star count
Forks
2,385
Using upstream fork count
Open Issues
0
Activity Score
0/100
0 commits in 30d
Created
Nov 14, 2023
Project creation date
README Summary
GPT-Crawler is a TypeScript tool that crawls websites to generate knowledge files for creating custom GPTs. It extracts content from web pages and converts it into a format suitable for training or feeding into GPT models. The tool allows users to build domain-specific knowledge bases from any website content.
AI Dev Skills
Unmapped
Web ScrapingData PreprocessingCustom GPT DevelopmentKnowledge Base CreationText Extraction and ProcessingData Pipeline Engineering
Tags
Web ScrapingData PreprocessingCustom GPT DevelopmentKnowledge Base CreationText Extraction and ProcessingData Pipeline EngineeringLocal DevelopmentDocumentation Processing for AI ModelsContent ManagementSelf-hostedTextEdTechCLI ToolDeveloper ToolsCustom GPT Training Data GenerationDomain-Specific AIKnowledge AugmentationContent Extraction for AI TrainingHTMLDomain-Specific Chatbot DevelopmentAI Training Data CreationWebsite Knowledge Base CreationKnowledge ManagementTypeScriptCLI
Taxonomy
Deployment Context
Skill Areas
Recent Activity
Updated 9 months ago
7 Days
0
30 Days
0
90 Days
0
Quality
prototype- Quality
- medium
- Maturity
- prototype
Categories
MLOps & InfrastructurePrimaryDev Tools & AutomationML Platform & InfrastructureEdge & Mobile AISearch & KnowledgeOther AI / MLFoundation ModelsModel Training
PM Skills
Scale & ReliabilityDeveloper Platform
Languages
TypeScript100.0%
Timeline
- Project created
- Nov 14, 2023
- Forked
- Mar 23, 2026
- Your last push
- 9 months ago
- Upstream last push
- 9 months ago
- Tracked since
- Jul 7, 2025
Similar Repos
pgvector cosine similarity · $0
Loading…