Rob Golding - Latest Comments

Re: Rob Golding: django-radius

Rob Golding — Sat, 28 Jan 2012 04:04:57 -0000

That's down to the customer's servers to look after. Django can't manage the users on external systems, it can just query them to see if a particular pair of credentials is valid or not.

Re: Rob Golding: django-radius

Tommyz4 — Sat, 28 Jan 2012 03:34:19 -0000

Interesting can you also manage the radius accounts through django ?

Re: Rob Golding: Django in Production: Part 3 - Automation & Monitoring

Matt Schick — Fri, 20 Jan 2012 16:55:38 -0000

Noticed that you run your production site right out of the cloned git repository. Has that ever caused any issue for you? I generally update my repository, copy out all of the files to a directory with the name of the current git commit, and update a symlink which points to the proper directory.

As you might imagine that creates a bunch of extra steps and places where things can go awry, but I've always done it because I've been paranoid about updating files that a wsgi server is currently running off of... is there any justification to my paranoia?

Re: Rob Golding: Django in Production: Part 2 - Background Tasks

Rob Golding — Fri, 20 Jan 2012 15:15:09 -0000

Thanks Suvash, I did mean to add that once I'd made the third post!

Re: Rob Golding: Django in Production: Part 2 - Background Tasks

suvash — Fri, 20 Jan 2012 07:10:43 -0000

would be great if you could also link the third article towards the end of the post. didn't realize there was one.

Re: Rob Golding: Django in Production: Part 2 - Background Tasks

Markus Gattol — Thu, 19 Jan 2012 16:35:27 -0000

django-ztask is yet another solution http://www.zeromq.org/story:3 and https://github.com/dmgctrl/dja...

Re: Rob Golding: Django in Production: Part 1 - The Stack

Kyle Hayes — Tue, 17 Jan 2012 11:05:09 -0000

Excellent article, Rob! I look forward to reading the other posts in the series. For me personally, an article like this has been desired for a long time. Thank you for the details.

Re: Rob Golding: Django in Production: Part 3 - Automation & Monitoring

Sergey Maranchuk — Mon, 16 Jan 2012 16:07:03 -0000

tnx, but what wrong with collectd compared to statsd?

Re: Rob Golding: Django in Production: Part 3 - Automation & Monitoring

Rob Golding — Mon, 16 Jan 2012 03:24:38 -0000

Thank you mate, in what capacity have you been using Graphite - for work or play? I actually found out about a Python version of statsd after spending hours installing Node.js on my EC2 instance!

Hope you're well too :)

Re: Rob Golding: Django in Production: Part 3 - Automation & Monitoring

Alistair Wooldrige — Sun, 15 Jan 2012 16:19:25 -0000

Excellent post Rob, I've been playing with Graphite recently, it's certainly a great tool! Not sure about statsd's lack of tests, although the concept is sound. Haven't come across Fabric before, will be delving into that this week, cheers!

Hope all is well,
Woolie

Re: Rob Golding: Django in Production: Part 2 - Background Tasks

Henk de Vries — Sat, 03 Dec 2011 16:23:24 -0000

Thanks for the info and sanity check ;-)

Re: Rob Golding: Django in Production: Part 2 - Background Tasks

Rob Golding — Sat, 03 Dec 2011 04:56:03 -0000

At the moment I have 2 worker processes running, using approx. 100MB in total. Given that, I'd say you're not doing anything wrong!

This is on an Amazon EC2 Micro instance with only 612MB of RAM in total.

Re: Rob Golding: Django in Production: Part 2 - Background Tasks

Henk de Vries — Sat, 03 Dec 2011 03:12:30 -0000

Thanks. I was trying to get a ballpark figure for celeryd alone. In a shared hosting environment memory usage is a (most often *the*) bottleneck and in my experience celeryd takes up to and over 100MB after some (at times big -- like reindexing a whoosh index w/ haystack) tasks have run. The initial memory usage, without any tasks, is already over 30MB (RSS). I'm hoping I'm doing something wrong.

Re: Rob Golding: Django in Production: Part 2 - Background Tasks

Ask Solem — Fri, 02 Dec 2011 19:02:49 -0000

The baseline for celeryd on my machine is 19MB for the main process, and 3.6MB for each pool worker process (set by the --concurrency setting). For django-celery this is 27MB for the main process with a basically empty project (Django does load a lot into memory, luckily a lot of this can be shared).

As Rob said, how much memory it consumes over time is subjective, depending on how much memory your tasks require when they run (and if you are unlucky, the memory leaks they come with, problems like these quickly appear in a long-running process environment, good we have workarounds, like the --maxtasksperchild option).
You can also enable an --autoscale setting to grow and shrink the number of processes depending on load, this
can be useful if you pay for memory/CPU-time.

Re: Rob Golding: Django in Production: Part 2 - Background Tasks

Rob Golding — Fri, 02 Dec 2011 15:58:59 -0000

That's a pretty subjective question. Celery isn't the only thing that's going to use memory - RabbitMQ/Redis should also be factored in.

The truth is it's all down to how busy the task queue is, and how intensive each task is. I'm working on a project at the moment which compiles LaTeX documents with Celery - and whilst that's not extremely heavy on RAM, the CPU requirements are quite large.

Re: Rob Golding: Django in Production: Part 2 - Background Tasks

Henk de Vries — Fri, 02 Dec 2011 14:08:30 -0000

Can I ask: how much memory does celery use (after running for, say, a couple of days)?

Re: Fixing Google Chrome’s 100% CPU Usage on Ubuntu

Chrome Rims — Thu, 01 Dec 2011 14:19:26 -0000

I try to create my own VPS on the ubuntu server, buy I am newbie.

Re: Rob Golding: Django in Production: Part 1 - The Stack

Tomas Cirip — Thu, 17 Nov 2011 19:01:09 -0000

Great post. Thanks for sharing. I use Apache, mod_wsgi, South and MySQL. Considering Nginx and Gunicorn.

Re: Fixing Google Chrome’s 100% CPU Usage on Ubuntu

Seattle chiropractor — Thu, 17 Nov 2011 17:11:35 -0000

I love my new Ubuntu, I didn't have any problem.

Re: Rob Golding: Django in Production: Part 1 - The Stack

Rob Golding — Thu, 17 Nov 2011 04:45:10 -0000

Thanks Matthew, you're right - if you want to use `python manage.py run_gunicorn` then it needs to be in your INSTALLED_APPS.

Re: Rob Golding: Django in Production: Part 1 - The Stack

Matthew Schinckel — Wed, 16 Nov 2011 20:24:48 -0000

You missed out that you need to add 'gunicorn' to your settings.INSTALLED_APPS

Re: Rob Golding: Django in Production: Part 1 - The Stack

Nicolas Grilly — Wed, 16 Nov 2011 18:28:01 -0000

Thanks for this useful post. I'm curious to know if you have any experience and/or opinion about uWSGI?

Re: Rob Golding: Django in Production: Part 1 - The Stack

Lucian Branescu — Wed, 16 Nov 2011 04:28:47 -0000

Same for us.

We're using Postgres largely because we trust it more: better durability (we've had corrupted myisam tables), less uncertain future.

Re: Rob Golding: Django in Production: Part 1 - The Stack

Rob Golding — Wed, 16 Nov 2011 04:26:30 -0000

Hi Chris,

Thanks for the feedback. Of course if you're using GeoDjango then PostGIS is a necessity :)

Glad you enjoyed the article.

Re: Rob Golding: Django in Production: Part 1 - The Stack

Chris — Tue, 15 Nov 2011 20:55:13 -0000

At Lo-Fi Art we use basically this exact stack on our servers. We do, as author noted is common, prefer Postgres. In our case it's a little more than personal preference as we run geodjango sites and Postgis is incomparable.

Good article - thanks!