Posts Tagged ‘Python’

« Older Entries
Newer Entries »

RSSmeme – Look out ReadBurner!

Wednesday, February 6th, 2008

Look out ReadBurner; RSSmeme is coming for you!  I registered rssmeme.com yesterday and launched the site today.  It was a test for me to see how good/fast of a Django developer I am.  It isn’t 100% done yet; but most of the functionality is there.

Why another one of these sites?  Because it was fun to make and competition is good.  I have things that ReadBurner doesn’t and ReadBurner has things that I don’t.  I think that eventually both of us will be better because of this.

How did I get so much data in less than 24 hours?  I got all the feeds by screen scraping ReadBurner (sorry if this offends you).  My update script parses through 50 feeds per minute; I’m not sure how fast ReadBurner is.

It isn’t the fastest site I’ve ever made; so please be gentle!  The caching is not so great yet.

If you have any requests please let me know.  I’ve kept things very simple so it’s easy for me to add new features at any time. I’m not sold on the color pallet yet either.

Tags: , , , , ,
Posted in Uncategorized | 1 Comment »

Beautiful Paginator with Django Custom Filters

Friday, February 1st, 2008

Here is an example; scroll down to see the paginator.

Django’s generic views already have awesome support for pagination.  But they don’t give you a great way of providing a good paginator; that is up to you to create.

They give you an array (page_range) in your templates that you can loop over to display a nice paginator but when the database gets full, do you really want to display every single page?  Google shows you 10 pages to start, and bumps that up to 20 later.  Flickr shows you some and then has a “…” with a final link to the last few.  I wanted to do something similar.

My first attempt involved looping over page_range and checking if each page was within 2 from the current page.  This worked alright, but I realized that once page_range grew to hundreds of pages I’d start seeing some slow down.

So instead I wrote a custom filter that will shrink page_range down to 2 pages before and 2 pages after the current page.  Refer to this post on how to make a custom filter.  Here is the code:

@register.filter
def shrink(value, arg):
    left = arg - 3
    right = arg + 2
    if left < 0:
        left = 0
        right = 5
    if right > len(value):
        right = len(value) + 1
        left = right - 6
        if left < 0:
            left = 0
    return value[left:right]

I’m sure someone can come up with a better way of doing that but it works very well for me.  Basically, if you pass in value of [1, 2, 3, 4, 5, 6, 7, 8, 9, 10] and an arg of 5 then it will return [3, 4, 5, 6, 7].  If you pass in 10 then it will return [6, 7, 8, 9, 10].  It will always give you an array containing at most 5 items and will try it’s hardest to center on the page you pass into it.

Spruce up the templates a bit by adding some padding to the links (so they are easier to click on), separating the First/Last/Next/Previous links from the numbers,  and don’t display unnecessary links and you’ve got one beautiful paginator.

Tags: , , ,
Posted in Uncategorized | No Comments »

Announcing Dear LazyWeb, a Twitter app!

Monday, January 28th, 2008

I’m proud to announce my latest project: Dear LazyWeb.  When somebody asks a question to nobody in particular on the internet they are calling for help from the LazyWeb.  There used to be a website where you could post your questions but it has been shut down.

Building applications on top of Twitter is all the rage these days.  It’s easy, convenient, and provides mobility; perfect for asking the LazyWeb.   Plus, people already use Twitter to ask the LazyWeb; but nobody is doing anything with all of that data. Enter dearlazyweb.

First follow dearlazyweb; dearlazyweb will follow you back in return.  Then send dearlazyweb a question.  You can do this through an @reply or a direct message:

@dearlazyweb What type of bear is best?

dearlazyweb will respond (via a direct message) with a url.  That url will lead you to a page with your question plus instructions on how to answer it.  Someone else might know the answer so they would respond:

 @dearlazyweb answer 1 Black

This time dearlazyweb will respond to you with a link to the answer.  Where did that number 1 come from? The page for each question contains instructions for answering it (it also happens to be the same ID as the Twitter message that created the question).  Now you know, black bears are best!  Someone else might disagree:

 @dearlazyweb a 1 Wrong, brown bears are better

Again, dearlazyweb will send you a direct message with a link to the answer.  Great, but doesn’t Twitter already provide this sort of communication?  Yes, but only those following you will be able to see your question and it won’t let you be anonymous.  Anyone can visit Dear LazyWeb to view questions so anyone can answer your question.  Plus, all direct messages sent to dearlazyweb are displayed anonymously.  So if you wanted to know the answer to a touchy subject:

d dearlazyweb Who are you going to vote for?

dearlazyweb will again give you a url; but this time there will be nothing at that url linking it to your Twitter account.  If someone responds to it publicly:

@dearlazyweb a 2 Isn’t it a little early to be deciding?

dearlazyweb will still direct message you with a link to that answer.  Only dearlazyweb knows that you created the question and it promises never to tell.  Answers can also be sent anonymously:

d dearlazyweb a 2 Obama!

dearlazyweb will direct message you with a link to this answer; only it will not contain any reference to the answerers Twitter screen name.

What if you want to shut dearlazyweb up?  You know the answer to your question; now it’s time to complete it:

@dearlazyweb complete 1

Done, you won’t hear a peep from dearlazyweb about that question ever again.  Even if someone else answers it; dearlazyweb won’t disturb you.

What if you would like to help someone out?

@dearlazyweb random

And dearlazyweb will send you a random, uncompleted question.  Help some poor lost soul out.

So, please, check out the site and the bot.  The site is full of extra information: it keeps track of how many questions and answers you’ve given and will even give you RSS feeds for individual questions.  Be sure to follow dearlazyweb or you won’t be able to receive any direct message.

Any feedback at all is appreciated.  I’m definitely not done developing the site or the bot so please let me know what you like or hate about either!

Tags: , , , , ,
Posted in Uncategorized | 6 Comments »

BeautifulSoup: Last.fm Artist Image

Wednesday, November 7th, 2007

A coworker had been working with Last.fm and libnotify to send him a notification when a friend listens to a song so that he can decide if he’d be interested in listening to it. To grab the artist’s image he used a complex regular expression that was prone to error. I thought it was a neat idea so I decided to implement for my stream. The result can be found here.

My implementation is a one liner:

event.image = soup.find('div', {'class': 'imgHolder'}).find('img').get('src')

Basically that says “find the div that has a class imgHolder, find the image inside of that div, and get me the link to the image”. BeautifulSoup makes things like this extremely easy. I highly recommend using it whenever you need to screen scrape.

Tags: , , , , , ,
Posted in Uncategorized | No Comments »

Pop Secrets Updated

Tuesday, November 6th, 2007

I wrote the original script for Pop Secrets over a year ago. Back then I was just getting into Python and was woefully ignorant. Using Universal Feed Parser I simply gobbled up all the content from a number of celebrity gossip blogs and spitted them out sorted by date onto a page. It was a complete hack; 129 lines of Python that was a complete mix of presentation, data, and logic. Despite all of that it still attracted a steady flow of about 250 unique visitors a month. Not bad but nobody was clicking on the Google ads and because the site consisted of a single static file I basically got 1 page view per visit.

Yesterday I rewrote Pop Secrets using Django, it took me a total of about 2 hours to finish and most of that was template design; Django is really that fast. From the outside the site looks the same except I’ve added pagination (3 posts per page) and you can view posts from a single source or tag. Internally I’m doing a lot more magic. I use BeautifulSoup to extract images from a post. This allows me to strip all the html from the content (don’t trust anything on the Internet!) but still keep the images (which is the main reason people visit these sites). Tags allow you to view all stories, across all sites, specific to a certain topic (Britney Spears for instance).

My logs show that people like the new version. I’ve gotten as many unique visitors in 1 day as I used to get in 1 month. People are making use of the tag and single source views. I’ve got an average of 18 page views per visit; people love clicking that “Next Page” button. I’m entertaining the idea of adding commenting; but I’m not sure people would use it since the gossip blogs already provide that feature.

If you came here from Pop Secrets: let me know what you think of it and if there is anything I can do to make it better for you.

Tags: , , , , , , , , , ,
Posted in Uncategorized | 4 Comments »

« Older Entries
Newer Entries »