Limiting the amount of RAM an application can use

Question

I am looking for a program that can limit the amount of RAM an application is allowed to use. If possible, free and compatible with Windows 7 x64.

Answer 1

Process Lasso provides some advanced process monitoring options:

http://www.bitsum.com/processlasso/index.php

You can configure rules inside a process watchdog that triggers an action (e.g. terminating the process) when the memory usuage becomes greater than defined.

Here is a screenshot:

You can also disallow specified processes from running, log all processes run, and even set various other process rules. These rules, amongst many others, include automatically restarting or terminating a process after it reaches some CPU or memory threshold