NetWare was designed around the 32-bit Intel i386 architecture which gives us 4 Gigabytes of space to operate in. Although NetWare can handle up to 64 GB of physical RAM, Intel's 32 bit architecture limits any OS to a 4 GB area for mapping Logical Memory. (The memory above 4 GB must be accessed by mapping pages in and out of the 4 GB space). Because most applications run in the "Kernel Space" or "Ring 0" in NetWare (as opposed to "User Space" or "Ring 3" in other operating systems), all NLMs running in the kernel have a finite amount of RAM to work with.
In NetWare prior to NetWare 5, the Traditional File System (TFS) was also NetWares memory manager. The TFS handled all memory allocated to file cache as well as all NLMs requesting memory allocation on the server. All memory requests were allocated from one physical memory pool, the File System Cache Pool (also known as the file cache buffers). And when you were low on file cache buffers, you simply added more memory to alleviate the problem.
But memory changed a lot in NetWare 5. For instance, Virtual Memory was introduced, as was a new memory manager called the NetWare Memory Manager. Another big change that occurred in NetWare 5.0 (and has been there ever since) was the introduction of three physical memory pools.
Because of these changes, memory management is not well understood in its present architectural model. And with the move of making NSS the default file system, the growing number of servers with 4 GB of physical memory, the ever-growing need of memory for NetWare services in each new Support Pack, and a bunch of other factors, many administrators are having memory issues. Because of the lack of memory information, system administrators are trying to place NetWare 4 memory solutions on NetWare 6 and above scenarios, with limited success.
Note: Even though NetWare 6.0 SP5 is on the end of life list, Novell will be coming out with a new SERVER.EXE for NetWare 6.0 SP5 to take advantage of some of the auto tuning features found in NetWare 6.5 SP3. There will be no more service packs for NetWare 6.0.
Memory Primer for NetWare 6.0 and 6.5:
As mentioned, a NetWare server has three logical address spaces.
- The File Cache Pool . This is the logical = physical range. This is where NSS runs and it is defined by the set parameter "File Cache Maximum Size".
- The user address space is used for Protected Mode Applications, including Java and ZENWorks apps, and can be controlled by the -u setting on the server.exe command line.
- The Virtual Memory (VM) Pool is the logical space where NLMs generally run, and it's size is 4GB minus the user space and File Cache Maximum Size.
Let's talk about the VM Pool:
The VM pool is now, and has been since 5.0, the main cache for all memory allocations except for NSS and a few NLMs which require logical = physical memory. The VM cache pool will fulfill all memory requests, even logical = physical requests when it has it. The VM cache pool also provides all stacks, NLM code and data, along with backing the virtual memory system.
If however the VM system can not provide the requesting NLM with the requested memory, then memory will be scavenged from the File Cache pool for use by the NLM. When the NLM is unloaded, or it reduces its memory requirements, the memory manager will assign only some of the memory back to the File Cache Pool, resulting in fragmentation of the File Cache Pool, and a reduction in available memory in the pool.
The VM Pool size is a derived figure: 4GB - File Cache Pool size - User Space, and will almost always be greater than installed memory on any server with less than around 2GB of physical memory.
This document takes you through Steps 1 to 6. I have been successful in resolving all memory related issues following this TID. Although keep in mind that it could take two or three months of tuning to really get a server tuned. Also keep in mind that each server behaves differently and needs to be tuned differently.
I'll go over a brief outline of the steps.
STEP 1: Update your server to the latest NetWare Support Pack. Update your server to the latest NSS files. The following steps require NetWare 6.0 SP5 or NetWare 6.5 SP2 and the latest NSS nlm's.
STEP 2 (If the module TSAFS.NLM is running on the server): TSAFS can be limited in the amount of cache it requests. To do this, unload the module TSAFS.NLM then re-load it with the following command-line switch:
Load TSAFS /CacheMemoryThreshold=1
In the file c:\nwserver\nssstart.cfg put the following lines:
To access this configuration help, open up NetWare Remote Manager (logging in as Admin), and click on "View Memory Config" in the left pane of the main window. From there, click on "Tune Logical Address Space." This opens a screen displaying configuration recommendations from the kernel developers at Novell. The recommended settings are calculated specific to the current server's running condition, and include information on how big to set the User Address Space size and the File System Cache Pool. (The NetWare kernel now stores the maximum amount of memory used by these pools over time, and can recommend optimal settings for them.) This will improve how the server uses memory because unused memory in one pool can automatically be given to the correct pool at boot time.
Remember the three main pools? 1. File Cache Pool, 2. User Address Space or UAS, 3. Virtual Memory or VM Pool.
The FILE CACHE MAXIMUM SIZE will move the line between the FS Cache Pool and the VM Pool up or down, depending on what you set the number to. The server - u parameter will move the line between the UAS and the VM Pool up or down, depending on what you set the number to. This is illustrated very effectively by going into the Memory Analyzer / SEG.NLM and examing the Advanced Summary Screen F10, then the Memory Mapping Screen F7. Or just look at the memory map in the SEGSTATS.TXT file.
One hint: If you have many servers and forget which memory tuning parameters have been done on which server, SEGSTATS.TXT is your answer. It will show a summary of all tuning that this document has discussed.
Novell Support has noticed that some customers are using the settings described in the steps above improperly for some configurations. We have included all 6 Steps above in an attempt to describe all possible factors contributing to or aggravating the problem of memory fragmentation. However, the inclusion of all these steps does not mean that every step is recommended for every customer. The steps outlined above should be followed in the order presented.
Controlling the memory that NLMs can use in the cache pool has proven to be successful with virtually all customers. Steps 2, 3 and 5 above detail three of the most prevalent examples of controlling memory used by specific NLMs. Other modules loaded on the NetWare server, from Novell or from 3rd party vendors, may require scrutiny and adjustments to regulate their role in consuming memory on the server. The NetWare Remote Manager (Module Listing) and other tools can be used to monitor memory consumption over time on a per-module basis on the server.
With the advent of NetWare 6.5 Support Pack 3, many important memory issues are addressed. These include the following:
- Memory fragmentation has been significantly decreased through algorithm changes that affect how logical memory address space is allocated.
- A new NSS API that flushes the NSS Cache Pool and allows the NetWare Memory Manager to change the File Cache Maximum Size line on the fly without rebooting the server. This only works if NSS runs with its cache balance set to On (the default), so unless told otherwise, leave NSS cache balance to its default of On.
- Support Pack 3 comes with memory auto-tuning enabled, so systems do not have the memory problems they experienced in Support Pack 2.
Support Pack 3 also looks at how much memory the server presently has. If, after the server has been running for a while, the NetWare OS needs to adjust the File Cache Maximum Size line to give more logical space for running NLMs, you will receive a message on the server console screen similar to the following:
"Server Logical Address Space is running low. The File Cache Maximum Size has been set to
If your auto-tune is set to OFF, you might be asked to make the changes yourself, in which case you will see a message on the server console screen similar to the following:
"Server Logical Address Space is running low. Increase the available logical space by increasing the File Cache Maximum Size to
With the auto-tune ON or OFF, you might be asked, in a message similar to the following, to increase the L!=P Address Space by shrinking the User Address Space:
"Server Logical Address Space is running low. Increase the available logical space by restarting the server with the -u