LAZ from laszip - corrupted display in CC

[Antoine]

Hello Daniel,
We are a lidar service provider company.
We have a client that often controls our LIDAR data using CC, and his statement is that CC uses liblas (PDAL now?) for decoding LAZ input files.
On our side We use directly laszip dll to read/write our LAZ we provide. laszip is also fully integrated in our workflow & softs.
Our client complaints because our data is degraded in CC,

>> screencapture in the next post message <<

the points are laid in row, which doesn't reflect the pattern of the MMS scanner (dual scanner, pattern should be cross-profil lidar)

>> screencapture in the next post message <<

Any idea if it comes from your lib decoder ?

Should you which any sample please ask.
thanks,
antoine

[Antoine]

display with CC (detail zoom)

display with CC (detail zoom).JPG (221.15 KiB) Viewed 7266 times

[Antoine]

display with laszip (detail zoom)

display with laszip (detail zoom)_.JPG (218.52 KiB) Viewed 7266 times

[Antoine]

On top of that display issue there is a pb in CC when reading the time scalar field...
Here again there seems to be another issue when CC reads time from source LAZ, that SF is corrupted during import...

SF-time display in CC.JPG (130.79 KiB) Viewed 7263 times

antoine

[daniel]

We still use libLAS (and LASlib by the way, you can test it by selecting the 'LAS 1.3 or 1.4' filter and see if it changes anything).

The issue your customer has, typically looks like an accuracy issue... just as if they ignored the Global Shift mechanism (and the associated warning ;). In CC we store the coordinates as 32 bits floating point values, which prevents from storing large coordinates 'as is'. The Global Shift mechanism is used to temporarily shift the coordinates in a local coordinate system (while the clouds are processed in CC), and they are restored at export time. This mechanism works great and prevents any loss of accuracy. It is also sufficiently smart to let the user load several clouds in the same 'local' coordinate system, to register the clouds with GCPs expressed in the global coordinate system, etc.

Apart from users periodically ignoring the warning message at loading time, I don't remember any other severe issue related to LAS import for a long time. However if you feel this is another issue, don't hesitate to send me a sample file.

For the GPS time scalar field, we have the same precision limitation (as the GPS time field is 64 bits). But in this case the field is automatically split into two fields, one with the 'high' part and one with the 'low' part. It may explain why the field looks funny. But once again, the fields are merged back at export time. I don't know which version you are using, but as this is something generally overlooked by most users, this may be a separate problem. Once again I think a sample file would be very helpful so that I can clear any 'suspicious' behavior from libLAS or the LAS importer.

[Antoine]

Daniel,
Do you have any private area where I could share that file...?
As for the time scale factor the extremum doesn't reflect what PDAL (or laszip) return...
antoine

[daniel]

You can send me any shared file to my email address (cloudcompare [at] danielgm.net). Otherwise I can send you a ftp account.

[Antoine]

sample sent.
Regarding the time SF let me just detail the kind of data we use. It's derived from MMS survey. TIme is one of the metadata info stored in our LAZ file. Most of the time in second of week GPS time.
Here the railtrack has been surveyed the same day, in 2 ways on both tracks.
Time SF should be ranged between 286500 and 303400. 1 way around 286500 and the other way around 303400.

PDAL - Time metadata

PDAL-time boundaries info.PNG (13.13 KiB) Viewed 7210 times

In CloudCompare after import the time SF is ranged by default between 0 and 16878 values...

time SF range in CC

Capture.JPG (51.36 KiB) Viewed 7210 times

It makes sens that CC shifts that physical info, 303400 - 286571 = 16900

time shift during import

Capture1.JPG (29.18 KiB) Viewed 7210 times

So Time SF looks ok. On that sample either I crop value around 0 to 6 (few secondes during the 1 way) or around 16900 (few secondes during the 2nd way)

antoine

[daniel]

Ok, I understand now what happens with the coordinates.

By default, CC uses for LAS file the internal 'LAS shift' (or 'offset') as a suggestion for its own 'Global Shift'.

In the case of your file, this 'LAS shift' is a bit surprising as it's very far from the actual coordinates:

cc_las_shift.png (14.99 KiB) Viewed 7204 times

The resulting local coordinates are still huge! (-5e5 for X and -9.0e6 for Y). I guess you have you reasons (very long tracks?), but this kills a bit the concept of the whole LAS shift mechanism. Especially since coordinates once the offset has been removed are stored on 4 bytes (i.e. 32 bits, just as CloudCompare does). You are playing with fire ;).

Anyway this also make fail our assumption that this LAS shift will indeed reduce the local coordinates to something that can be safely stored in 32 bits floating point values. As a result (even though the coordinates do appear in red to indicate that there's an issue ;), if the users continue and clicks 'Yes', then the precision is clearly not on the safe side and they will definitely observe some rendering glitches as OpenGL doesn't like big numbers as well...

To avoid this, they have to manually fix the translation to get smaller coordinates:

cc_las_shift2.png (15 KiB) Viewed 7204 times

[daniel]

For the 'Time' scalar field, I remember now that we indeed shift the time values with the smallest time found in the file so as to avoid accuracy issues as much as possible (we don't split the fields anymore).

In your example you can easily separate both days by using the 'Edit > Scalar fields > Filter by value' tool.