Which version and edition of SQL server are you using ?

Are you using windows 2000 32 or 64 bit ?

How many CPU ?

What kind of disk system do you have ?

-Sat 

SQL Server 2000 - Enterprise Edition -  SP4

Windows 2003 - SP2

8 CPUs

RAID5

If you are running on windows 2003 32 bit are you running with PAE  on windows and AWE on  SQL SERVER settings ?

This helps utilize more RAM but if you are on 64 bit , you may have to dig dipper like check perfmon counters for any abnormal figures on memory , disk subsystem and SQL batches\Sec.

Also if this issue is happening only after more users are logged on and the users are hitting IIS first which in turn talks to SQL server probably you need to check APP server as well for any pool memory issues.

-Sat 

Yes, have the /PAE switch on both nodes. AWE enabled is enabled too.

I have set a whole lot of performance counters, don't see anything abnormal there.

"User Connections" are extremely high, more than 300 at the time the server hangs.

Yes, users first hit IIS which talks to SQL Server. When you say check the app server for "pool memory issues" are you saying use the 'Memory' performance counters on those servers to monitor? 

Here is part output of dbcc memorystatus (Converted to MB/GB):

Stolen                   20MB
Free                     173MB
Procedures            663MB
Inram                    0
Dirty                     789 MB
Kept                      0
I/O                        0
Latched               138
Other                  18.17GB

Commited                     19.78GB
Target                           19.78GB
Hashed                         18.94GB 

Thanks.