Hacker News new | past | comments | ask | show | jobs | submit login

What if the content you want to scrape->API is behind a login gate? Is there an option of authentication?



You can actually write an API endpoint that'll retrieve a state token (which includes the cookies). For one example, you can use view our Hacker News login endpoint at https://wrapapi.com/#/view/phsource/hackernews/login/latest

That endpoint will then emit a state token, which includes the session cookies. You can feed that state token into your next request and it'll authenticate you


Yes, you can get the content you want even if it's behind a login page! Expect to create 2 APIs, one for logging in and another for getting the content. An example is provided on their homepage: https://wrapapi.com/v2#/caseStudies/cj




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: