Hey guys, first let me start by saying, what a fantastic program!
I think cloud compare has a lot of future in the point cloud world.
My question is what is the limit of points that the program can be handled?
Because I tried a .pts with 210,000,000 points in two computers, one in a 8 core 4g ram – windows xp64
and the second one in a 16 core 16g ram and had the same error (not enough memory) in the 83% range. I tried a couple of other heavy clouds but it seems that cloud compare has a limit on handling certain amounts of points.
Is this true? Or is there something that I’m doing wrong?
Please anybody that knows or has had the same problem please your input is well appreciated.
Question
Re: Question
Very interesting question!
CloudCompare keeps an old limit that was meant for 32 bits systems with 2Gb memory. It only handles clouds of less than 128,000,000 points. The main reason of this limitation was that if you are trying to load 'only' 128 M. points (without color or normal), you need 3 x floats = 12 bytes per point. About 1.43 Gb, just for blank points (so, if you add to this the system load, you are generally full on Windows XP with 2Gb ram). Above this limit, CC splits the file in several clouds. However, I haven't tested this for a long time (I haven't enough memory on my own computers, nor a 64 bits OS ... shame on me). Maybe an issue appeared meanwhile. I'll try to find a computer with a 64 bits system and at least 4 Gb.
I have two questions regarding your issue:
- did you use the official CC release downloaded on the website? (it is compiled with a 32 bits compiler, so I'm not sure you can go above the 3.6Gb limit with it, even on a 64bits version of Windows)
- were you loading a blank cloud or also colors, normals, etc.?
I know that some users have compiled CC successfully on Linux 64 bits, and I guess it should be possible to do it also on Windows 64 bits (in particular, there is now a Visual 2008 project shipped with the sources that should help in this way). I wonder if someone has tried to load so many points with such versions?
Anyway, other parameters must be taken into account:
- displaying and interacting with so much points in the actual CC version would be very 'heavy'. Prior to this we need for instance an octree structure designed for interactive LOD (level of detail) management/display and dedicated to this task. The octree actually used in CC is optimized for distance computation (it's structure and implementation doesn't allow fast LOD approaches in particular). We would also need pre-processes to correctly handle such big clouds (like what is done for out-of-core memory management for instance).
- the actual octree implementation (once again) is not meant for processing more than ~40,000,000 points (it has a maximum depth of 10 levels of subdivision). It requires also 8 bytes per point (so about 1 Gb for 128,000,000 points). You can however recompile CC to handle up to 21 levels of subdivision (see below) but I'm quite sure it won't be so easy to get all the algorithms to work nicely after this ...
Daniel
More technical details:
- in "ccPointCloud.h" (qCC project), there's a constant named CC_MAX_NUMBER_OF_POINTS_PER_CLOUD that should be changed to allow clouds of more than 128 M. points (we should transform this to an application parameter)
- in "CCTypes.h" (CCLib projet), there's a compiler macro named OCTREE_CODES_64_BITS that should be defined in order to handle octrees with 21 levels of subdivision (however, I haven't tested this for a long time)
- Theoretically, we can store up to 4 Gb points (!) as we slice the memory in chunks of 65536 points (to avoid issues such as finding sufficiently big contiguous memory blocks). But storage is a minor problem compared to all the others I have mentioned before.
CloudCompare keeps an old limit that was meant for 32 bits systems with 2Gb memory. It only handles clouds of less than 128,000,000 points. The main reason of this limitation was that if you are trying to load 'only' 128 M. points (without color or normal), you need 3 x floats = 12 bytes per point. About 1.43 Gb, just for blank points (so, if you add to this the system load, you are generally full on Windows XP with 2Gb ram). Above this limit, CC splits the file in several clouds. However, I haven't tested this for a long time (I haven't enough memory on my own computers, nor a 64 bits OS ... shame on me). Maybe an issue appeared meanwhile. I'll try to find a computer with a 64 bits system and at least 4 Gb.
I have two questions regarding your issue:
- did you use the official CC release downloaded on the website? (it is compiled with a 32 bits compiler, so I'm not sure you can go above the 3.6Gb limit with it, even on a 64bits version of Windows)
- were you loading a blank cloud or also colors, normals, etc.?
I know that some users have compiled CC successfully on Linux 64 bits, and I guess it should be possible to do it also on Windows 64 bits (in particular, there is now a Visual 2008 project shipped with the sources that should help in this way). I wonder if someone has tried to load so many points with such versions?
Anyway, other parameters must be taken into account:
- displaying and interacting with so much points in the actual CC version would be very 'heavy'. Prior to this we need for instance an octree structure designed for interactive LOD (level of detail) management/display and dedicated to this task. The octree actually used in CC is optimized for distance computation (it's structure and implementation doesn't allow fast LOD approaches in particular). We would also need pre-processes to correctly handle such big clouds (like what is done for out-of-core memory management for instance).
- the actual octree implementation (once again) is not meant for processing more than ~40,000,000 points (it has a maximum depth of 10 levels of subdivision). It requires also 8 bytes per point (so about 1 Gb for 128,000,000 points). You can however recompile CC to handle up to 21 levels of subdivision (see below) but I'm quite sure it won't be so easy to get all the algorithms to work nicely after this ...
Daniel
More technical details:
- in "ccPointCloud.h" (qCC project), there's a constant named CC_MAX_NUMBER_OF_POINTS_PER_CLOUD that should be changed to allow clouds of more than 128 M. points (we should transform this to an application parameter)
- in "CCTypes.h" (CCLib projet), there's a compiler macro named OCTREE_CODES_64_BITS that should be defined in order to handle octrees with 21 levels of subdivision (however, I haven't tested this for a long time)
- Theoretically, we can store up to 4 Gb points (!) as we slice the memory in chunks of 65536 points (to avoid issues such as finding sufficiently big contiguous memory blocks). But storage is a minor problem compared to all the others I have mentioned before.
Daniel, CloudCompare admin