Relevant Search on the Web with a Personal Crawler

I thought (and I strongly believe that somebody already thought about the same before me :D) that it would be great to have a search that’s relevant only to you. With Google you can kind of personalize your searching experience, but I mean another thing.

Let’s say:

  • I’ve got hundreds of followers on Twitter and I follow a hundred people
  • I’m on Facebook where I’ve got many friends
  • I’m using Delicious for bookmarks
  • I use Google Reader for keeping myself up to date
  • I blog myself, and I receive comments on my posts
  • etc.

In all the listed services/activities there is a substantial set of links present or produced either by me or by people in my network. These links form a slice of Internet that is relevant to me.

Intuitively, the part of Internet that is relevant to me is growing:

  1. as I’m expanding my social graph
  2. as I’m sharing links with others
  3. as I’m actively participating on the Web, e.g. commenting

however, a big part of my subInternet is actually hidden (not public), e.g. links shared by friends on Facebook.

Now, imagine that you have your personal Web crawler (maybe plugged into Google). This way you could search in 3 modes:

  1. The Web – as we are used today
  2. The relevant part of the Web only – available via OAuth, Facebook Connect, etc.
  3. Both

As an example consider the keyword “birthday” in all the three cases. It’s pretty clear that the results will be quite different.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: