Process multiple files at once

To post any request/idea for new functionalities
PablerasBCN
Posts: 296
Joined: Sat Jan 20, 2018 1:57 pm

Process multiple files at once

Post by PablerasBCN »

Hi, I've to load lets say 100 clouds and would like to run AO on all of them and would like to convert their luminance PCV scalar field ro RGB all in teh same process and save them all to a folder at the same time. For instance.

As it is now, hwen you multi select clouds commands do get dissabled, please add this, I believe should be fairly easy.

Not even a batch queue simply multi select have enabled commands so it runs with same params on all selected clouds.
PablerasBCN
Posts: 296
Joined: Sat Jan 20, 2018 1:57 pm

Re: Process multiple files at once

Post by PablerasBCN »

checking the holly todo list found

[*] Run the PCV plugin on more than one file at a time (or batch process)

ok my post was bit redundant then, but the multiple process thing could be extended to the other operations I did mention like "bake" scalar fields and such.
PablerasBCN
Posts: 296
Joined: Sat Jan 20, 2018 1:57 pm

Re: Process multiple files at once

Post by PablerasBCN »

Ok while this is not the perfect solution for an overnight or to let over the weekend cause is not a 1 by 1 endless processing, yet, found looking at the task manager when computing AO, the CPU mem usage and GPU does not go too high, so opening like 5 instances of CC and loading on each a different cloud, you run AO so the CPU gets way bussier like 80% instead of the 15% for a single instance. This way after finding the correct chunk size for the max available rays and resolution I can run multiple "renders" and is not as messy as would be to have multiple loaded at a time cause on each instance you work with only a cloud so SF->Bake to clor map, save and wipe. Untill something like a batch is implemented this helps.
daniel
Site Admin
Posts: 7710
Joined: Wed Oct 13, 2010 7:34 am
Location: Grenoble, France
Contact:

Re: Process multiple files at once

Post by daniel »

Ok, I modified the PCV plugin so that it can process multiple files in a batch (one after the other). It's already available in the online 2.10.alpha version.
Daniel, CloudCompare admin
PablerasBCN
Posts: 296
Joined: Sat Jan 20, 2018 1:57 pm

Re: Process multiple files at once

Post by PablerasBCN »

AWESOME! TX!!
PablerasBCN
Posts: 296
Joined: Sat Jan 20, 2018 1:57 pm

Re: Process multiple files at once

Post by PablerasBCN »

This may just have been a concidnce but I managed to crash CC testing the new functionality XD

See attached image to understand the dataset:
https://1drv.ms/i/s!AuliwwO_siw1imVg_cLMawWaw21x

This dataset had 80 parts, some of wich where really small, probably the dataset is not properly processe beforehand as I was finding some weird stuff. One of the original cells I have it merged so there where all kind of sizes. Yet, I had the system monitor opened and memory did not peak or anything.

I had in paralel running a 16 cell dataset running too. and that is doing the job as spected. May have just been a coincidence. Will report future crasehs if any.
PablerasBCN
Posts: 296
Joined: Sat Jan 20, 2018 1:57 pm

Re: Process multiple files at once

Post by PablerasBCN »

Second one also crashed :\
I was not on the screen to see the memory usage and such
https://1drv.ms/i/s!AuliwwO_siw1imZhNZdgfZcjBBhz

And forgot to comment but It felt like it could be slower but I'll download previous version and check that out
PablerasBCN
Posts: 296
Joined: Sat Jan 20, 2018 1:57 pm

Re: Process multiple files at once

Post by PablerasBCN »

K definitely looks like there is an stability problem:
Crashed again with different dataset:
https://1drv.ms/f/s!AuliwwO_siw1iml7kp_m7xqD0htl

I see no log so choose to debug no idea if the error in VS is due to VS trying to acces the dll or is the eror msg itself. hop it helps.

DATASET link (public data):
https://b5m.gipuzkoa.eus/lidar/LQ13_12_mo.zip
daniel
Site Admin
Posts: 7710
Joined: Wed Oct 13, 2010 7:34 am
Location: Grenoble, France
Contact:

Re: Process multiple files at once

Post by daniel »

Ah! I found the bug: it was occurring when in the selection there are non elligible entities (groups, etc.). Not sure why, but in this case it would make the plugin crash. I'll update the online version asap.
Daniel, CloudCompare admin
daniel
Site Admin
Posts: 7710
Joined: Wed Oct 13, 2010 7:34 am
Location: Grenoble, France
Contact:

Re: Process multiple files at once

Post by daniel »

Ok I've updated the 2.10.alpha version. Thanks for the feedback!
Daniel, CloudCompare admin
Post Reply