Some Tips for working with large raster files

Hi everybody

I know, I was in the middle of talking about Van-Genuchten model somehow, but I thought It's better to just say what I have in mind before I forget it.

In the past few weeks I have been working on one project, which needed me to first provide some huge raster files (Larger than 10 GB). I faced with some problems that they put a delay on my work. In general there is not too much to say for now, it's just few number of tips (probably I will add some more things later on) :

1- Whenever you want to work with very large Raster files, first build the pyramid of your ratser. According to the ESRI website (http://webhelp.esri.com/arcgisdesktop/9.3/index.cfm?TopicName=Raster_pyramids) "Pyramids are reduced resolution representations of your dataset used to improve performance".

ArcMap and ArcCatalog generally ask you to build a pyramid by default, so please listen to them.

2-I have tried different formats with different compression types for exporting a one raster file. Tiff format with LZW compression type worked great for my large raster files.

Here is brief description of LZW method :

"LZW is named after Abraham Lempel, Jakob Ziv and Terry Welch, the scientists who developed this compression algorithm. It is a lossless ‘dictionary based’ compression algorithm. Dictionary based algorithms scan a file for sequences of data that occur more than once. These sequences are then stored in a dictionary and within the compressed file, references are put where-ever repetitive data occurred."

Fianlly, since TiFF is able to support up to 32 bit depth and it supports different compression methdos, I think it's worth it to try and see how it works on your prject :)