PHP - building a scraper - find final destination of redirected URL?

[Icecube]

hi guys

this scraper should parse pages where links to external sites are served through a redirection
I mean, to link example.com Web Page they'll send you to http://link.thissite.com/some-weird-url&anykind=ofcrap, that redirects you to example.com

how can I find out where it redirects?

suggestions,link,guides,whatever is welcome

should it be done with http headers?
it's just the only way I can imagine, but I've never worked with headers so, again, any suggestion is welcome

 

[Juicify]

If you use curl you can
curl_setopt($curl, CURLOPT_FOLLOWLOCATION, true);

 

[Icecube]

looks exactly what I need

say I want to store the last location in some variable...how would I do it?

will it put the last location in the scraped array instead of the plain scraped URL? sounds too good to be true...

I am learning curl, I've just read smaxor's tutorials so I can't say I can really use it

 

[tobsn]

server reponse headers is the solution.

you have to grep for "Location: (.+?)\r\n" - if you're using file_get_contents use

$http_response_header

thats an array iirc that will get filled by php after the request.

:error:

 

[wesley]

 curl_getinfo($ch, CURLINFO_EFFECTIVE_URL);

 

[Icecube]

uhm...I think I have enough stuff to try
thank you guys

 

[georgiecas1y]

 curl_getinfo($ch, CURLINFO_EFFECTIVE_URL);

learn something new every day, cheers

 

[nis]

Do you really need the last URL? When using FOLLOWLOCATION curl will return the last page in any redirection-chain, which is what I presume is what you are after when talking about a scraper.

 

[Icecube]

I am not sure which one I should use, I need to store the final site where the user lands when clicking that link, I don't care through which hell of redirections he's going through

I didn't try the stuff I have been suggested yet

 

[wesley]

The solution has already been spelled out for you. If you use CURL, use my solution. Otherwise, test the other methods given.

 

[nis]

I didn't try the stuff I have been suggested yet

Guess what you need to do

 

[Icecube]

I just have had no time to test the stuff guys, thank you a lot for all the suggestions