Basically the differences are:

1. Use urlfetch instead of urllib2
2. Use Django’s urlencode which will work with unicode (letting you comment/share in languages like Chinese)
3. Use demjson instead of cjson (which cannot be used on Google App Engine), simplejson, or json (which both throw decoding errors from time to time)

Enjoy!

Update:

You don’t need to use demjson anymore; simplejson will work just fine.  I just got an email from Sanjeev saying that they fixed the issue (’\x’ is not valid JSON).  This makes things even easier because simplejson is included in Django (from django.utils import simplejson) which is included in Google App Engine.

You know when you see someone’s Twitter update with an @ reply in it to someone you don’t know and are completely confused about the topic?  Tweet 2 Tweet is an effort to solve that.  You can put both of their names into Tweet 2 Tweet and see their conversation on one page much like Facebook’s “Wall-to-Wall” feature.

It’s just a dumb, fun project.  Nothing made to be a big splash; just something I’ve been meaning to do.  It uses Django + App Engine + Summize to power it.  Django mainly for the templates, App Engine so my server doesn’t get anymore overloaded, and Summize so I don’t even need a database.

Plus it was an excuse to whip out my old algorithms book so I could implement a quicksort.

Thanks to everyone who’s covered it already; I’ll be making minor improvements from time to time.

On March 21st Bret Taylor (one of the FriendFeed founders) asked if I’d like to provide feedback for the then upcoming FriendFeed API.  I was very interested in getting my hands on it and stoked that I was one of the people he thought of when searching for feedback.  Over the weekend I tinkered with the API, provided feedback (you can thank me for having comment/like/discussion and search feeds in the API), and built the first FriendFeed application: FriendFeed Stats.

On March 25th I launched FriendFeed Stats simultaneously with the release of the FriendFeed API.  Soon I open sourced the Django application that drives FriendFeed Stats so everyone could see how simple it is to work with the FriendFeed API.

Many thanks to Bret and the rest of the FriendFeed team for delivering an awesome service.  I hope to see many more interesting FriendFeed applications soon.

http://www.rssmeme.com/feeds/search/yahoo/google/microsoft/

Tada!

Why another one of these sites?  Because it was fun to make and competition is good.  I have things that ReadBurner doesn’t and ReadBurner has things that I don’t.  I think that eventually both of us will be better because of this.

How did I get so much data in less than 24 hours?  I got all the feeds by screen scraping ReadBurner (sorry if this offends you).  My update script parses through 50 feeds per minute; I’m not sure how fast ReadBurner is.

It isn’t the fastest site I’ve ever made; so please be gentle!  The caching is not so great yet.

If you have any requests please let me know.  I’ve kept things very simple so it’s easy for me to add new features at any time. I’m not sold on the color pallet yet either.

Django’s generic views already have awesome support for pagination.  But they don’t give you a great way of providing a good paginator; that is up to you to create.

They give you an array (page_range) in your templates that you can loop over to display a nice paginator but when the database gets full, do you really want to display every single page?  Google shows you 10 pages to start, and bumps that up to 20 later.  Flickr shows you some and then has a “…” with a final link to the last few.  I wanted to do something similar.

My first attempt involved looping over page_range and checking if each page was within 2 from the current page.  This worked alright, but I realized that once page_range grew to hundreds of pages I’d start seeing some slow down.

So instead I wrote a custom filter that will shrink page_range down to 2 pages before and 2 pages after the current page.  Refer to this post on how to make a custom filter.  Here is the code:

@register.filter
def shrink(value, arg):
    left = arg - 3
    right = arg + 2
    if left < 0:
        left = 0
        right = 5
    if right > len(value):
        right = len(value) + 1
        left = right - 6
        if left < 0:
            left = 0
    return value[left:right]

I’m sure someone can come up with a better way of doing that but it works very well for me.  Basically, if you pass in value of [1, 2, 3, 4, 5, 6, 7, 8, 9, 10] and an arg of 5 then it will return [3, 4, 5, 6, 7].  If you pass in 10 then it will return [6, 7, 8, 9, 10].  It will always give you an array containing at most 5 items and will try it’s hardest to center on the page you pass into it.

Spruce up the templates a bit by adding some padding to the links (so they are easier to click on), separating the First/Last/Next/Previous links from the numbers,  and don’t display unnecessary links and you’ve got one beautiful paginator.

Building applications on top of Twitter is all the rage these days.  It’s easy, convenient, and provides mobility; perfect for asking the LazyWeb.   Plus, people already use Twitter to ask the LazyWeb; but nobody is doing anything with all of that data. Enter dearlazyweb.

First follow dearlazyweb; dearlazyweb will follow you back in return.  Then send dearlazyweb a question.  You can do this through an @reply or a direct message:

@dearlazyweb What type of bear is best?

dearlazyweb will respond (via a direct message) with a url.  That url will lead you to a page with your question plus instructions on how to answer it.  Someone else might know the answer so they would respond:

 @dearlazyweb answer 1 Black

This time dearlazyweb will respond to you with a link to the answer.  Where did that number 1 come from? The page for each question contains instructions for answering it (it also happens to be the same ID as the Twitter message that created the question).  Now you know, black bears are best!  Someone else might disagree:

 @dearlazyweb a 1 Wrong, brown bears are better

Again, dearlazyweb will send you a direct message with a link to the answer.  Great, but doesn’t Twitter already provide this sort of communication?  Yes, but only those following you will be able to see your question and it won’t let you be anonymous.  Anyone can visit Dear LazyWeb to view questions so anyone can answer your question.  Plus, all direct messages sent to dearlazyweb are displayed anonymously.  So if you wanted to know the answer to a touchy subject:

d dearlazyweb Who are you going to vote for?

dearlazyweb will again give you a url; but this time there will be nothing at that url linking it to your Twitter account.  If someone responds to it publicly:

@dearlazyweb a 2 Isn’t it a little early to be deciding?

dearlazyweb will still direct message you with a link to that answer.  Only dearlazyweb knows that you created the question and it promises never to tell.  Answers can also be sent anonymously:

d dearlazyweb a 2 Obama!

dearlazyweb will direct message you with a link to this answer; only it will not contain any reference to the answerers Twitter screen name.

What if you want to shut dearlazyweb up?  You know the answer to your question; now it’s time to complete it:

@dearlazyweb complete 1

Done, you won’t hear a peep from dearlazyweb about that question ever again.  Even if someone else answers it; dearlazyweb won’t disturb you.

What if you would like to help someone out?

@dearlazyweb random

And dearlazyweb will send you a random, uncompleted question.  Help some poor lost soul out.

So, please, check out the site and the bot.  The site is full of extra information: it keeps track of how many questions and answers you’ve given and will even give you RSS feeds for individual questions.  Be sure to follow dearlazyweb or you won’t be able to receive any direct message.

Any feedback at all is appreciated.  I’m definitely not done developing the site or the bot so please let me know what you like or hate about either!

My implementation is a one liner:

event.image = soup.find('div', {'class': 'imgHolder'}).find('img').get('src')

Basically that says “find the div that has a class imgHolder, find the image inside of that div, and get me the link to the image”. BeautifulSoup makes things like this extremely easy. I highly recommend using it whenever you need to screen scrape.

Yesterday I rewrote Pop Secrets using Django, it took me a total of about 2 hours to finish and most of that was template design; Django is really that fast. From the outside the site looks the same except I’ve added pagination (3 posts per page) and you can view posts from a single source or tag. Internally I’m doing a lot more magic. I use BeautifulSoup to extract images from a post. This allows me to strip all the html from the content (don’t trust anything on the Internet!) but still keep the images (which is the main reason people visit these sites). Tags allow you to view all stories, across all sites, specific to a certain topic (Britney Spears for instance).

My logs show that people like the new version. I’ve gotten as many unique visitors in 1 day as I used to get in 1 month. People are making use of the tag and single source views. I’ve got an average of 18 page views per visit; people love clicking that “Next Page” button. I’m entertaining the idea of adding commenting; but I’m not sure people would use it since the gossip blogs already provide that feature.

If you came here from Pop Secrets: let me know what you think of it and if there is anything I can do to make it better for you.