Overview
Manual tracking and formatting of Hacker News job posts wastes time and invites errors. This automation scrapes the latest "Ask HN: Who is hiring" posts, cleans and parses recruitment comments with an LLM, then batches structured job data directly into Airtable—cutting manual work and enforcing data consistency.
The Impact
- Automate data capture. Eliminate manual scraping by auto-fetching fresh job listings from Hacker News.
- Clean and parse text. Remove HTML noise and extract structured job info using an LLM for reliable records.
- Batch upload to Airtable. Save parsed jobs efficiently in your database with minimal API calls.
- Ensure data quality. Filter out invalid entries to keep your job records accurate and usable.
Who This Is For
- Recruitment teams who need automatic, up-to-date job listings for internal tracking.
- Community managers curating open-source or tech job boards from Hacker News posts.
- Data analysts studying talent market trends via structured job data aggregation.
- Developers and automation engineers seeking to streamline job data workflows.
How It Works
- Fetch Latest Posts
- Query Algolia API for "Ask HN: Who is hiring" posts, filter recent (last 30 days) and select the latest.
- Retrieve Post Details
- Call Hacker News API to get main post and extract comment IDs, limiting to MaxJobs count.
- Scrape and Clean Comments
- Loop through comment IDs, fetch each comment, strip HTML entities and tags to produce clean text.
- Parse Structured Job Data
- Send cleaned comment text to LLM to extract JSON with fields like company, title, location, type, salary, and URLs.
- Prepare and Save Records
- Aggregate parsed results, filter valid entries, then write batch records to Airtable and output summary.
What You'll Need
Before using this template, make sure you have:
- An Airtable Base and a Table with fields: company, title, location, type, work_location, salary, description, apply_url, company_url.
- A valid Airtable API credential authorized to write records into your target Base and Table.
- An Algolia API key with search-only access for the Hacker News "hn" application (used as x-algolia-api-key header).
- IDs for your Airtable Base (appXXXXXXXXX) and Table (tblXXXXXXXXX) to configure the workflow inputs.
How to Use
- Step 1. Setup Airtable
- Step 2. Configure Credentials
- Step 3. Provide API Keys and IDs
- Step 4. Set MaxJobs Limit
- Step 5. Run and Verify
Create your Airtable Base and Table with the required fields for job data storage.
Link your Airtable API credentials in the workflow to enable record writing.
Input your Algolia API key, Airtable Base ID, and Table ID into the workflow parameters.
Adjust how many job comments to process per post to control workload and testing scope.
Execute the workflow and confirm the parsed job records appear correctly in your Airtable.
