The texture atlas is texture pack that's generated from the UI textures so that they can be loaded into memory more quickly and helps reduce texture swapping. By increasing the size of the texture atlas, you could further reduce texture swapping, if your video card supports textures that large. If you don't know how large textures your video card supports it's probably best not to play with that setting.
