Noob question about other people databases's

[1337hovie]

I apologize in advance if this is a really stupid question, flame suit on.

So i want to write a script that can like..scrape results from other people's DB searches. (which i know is possible, however not my question)

Think real estate, so people can use my search to pull those new featured listings from all over the US and list results...but on my site, pulling real estate databases from each state.

I guess my question is, can i just ask them (them being the owners of the databases) for SELECT abilities? does that normally happen? (is that something a web admin would laugh at?) do i have to pay for something like that?

am i better off just scraping them?

Thanks fellas,
jt

 

[Rexibit]

I guess my question is, can i just ask them (them being the owners of the databases) for SELECT abilities?

You can, they just need to make you a new user permission.

does that normally happen?

Not normally, but if you know a realtor that runs their own listings, then they will probably let you if you can offer them value for doing it.

(is that something a web admin would laugh at?)

If they don't know you, probably.

do i have to pay for something like that?

If you can offer them a good reason for doing it and prove that they'll benefit, then no.

am i better off just scraping them?

Depends on the data you want and the accuracy.

Personally, you should go to the source. Most Realtors just buy into a suite/service that will show their listings for them and tie them in with other Realtors from multiple cities so that they can show their customers multiple properties without having to send them somewhere else.

Here's a source, they even have their own iPhone app and API. Real Estate Data, Mortgage Data, API - Zillow Developer Tools

Oh, and I got this from Googling "US real estate API", easy huh?

 

[1337hovie]

I appreciate your response, and answering all of my questions individually - very thorough of you. Didn't think i get more than 2 words.

the niche isn't real estate, its something similar. it's gov't related. (should i not be scraping gov't sites? haha). almost like a database of everyone who's been in debt. (not debt, but its public information).

what if the data isn't updated very often? its pretty much entered in, and left for a long time (i would imagine new records would be added, but once i scrape that record - i wouldnt need it again).

thanks again

 

[gravearc]

Your post is pretty confusing, but I'll try to answer it.

If someone enters a search term on your form, you can save that easily.

You can't just raw SQL statements on someone's database without hacking it. No one will give you that access

What you can do to get data is find out how they manage their presentation of it and exploit that.
e.g. if site.com/ has a form that provides info when you enter a city, you can make a script that will enter 500 town names and scrape the results for each one.

If you PM me what you're trying to do, I'll take a look at it.

 

[1337hovie]

Yeah its pretty much down to scraping i guess, i doubt ill be able to have access to their db's. and each db varies on the state i'd imagine, being operated/managed seperately.

so exploiting any form GET methods, and things like that are my best bet?

Thanks guys, sorry i cant go into more detail - so i understand if advice is limited.

 

[gravearc]

If they send info when you make a request, what more access do you need. It doesn't have to be get.