-
Notifications
You must be signed in to change notification settings - Fork 9
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Crash when trying to convert a vpc into a raster with raster_tin #31
Comments
Note: when using to_raster instead of to_raster_tin, it usually goes a bit further but eventualy also crash. I noticed that all cpus are used at 100%, which is probably not good as I can't even log on my machine from another terminal.. |
hi @flemmens I think the problem in your case will be that you run out of memory and the pdal_wrench process gets killed. By default, pdal_wrench runs as computation in multiple threads - as many as the number of logical CPUs. The raster export via TIN is quite memory intensive, so try limiting the number of threads with e.g. Bigtiff should not be a problem - the raster creation is done by GDAL which will use bigtiff automatically if needed. As for |
Thank you for your reply @wonder-sk, I was not aware about this --threads option. I made a new test with a small VLC of 219 laz files (24Gb) and ran the process on a 96Gb RAM computer with 48 cores using only 4 threads. The process last a lot longer but finaly crashes at 30%, so it still does not work. However, the subtiles created in the temporary directory are OK, I was able to join them using gdal and got a GeoTIFF of about 1/3 of my terrain. All the .laz used for the test have been checked, I can process them with a classic pdal pipeline without having any issue. If you wish do the test by yourself I can provide a S3 link with the files. |
Hello,
I would like to report an issue:
I have a bunch of .laz files which I would like to convert into a raster.
To begin I create a vpc, which works fine:
Then I want to convert the vpc into a raster tif using the raster_tin method:
If I have only a few .laz files in my-laz-tiles.txt, everything works well.
However, if I have more files (let's say 100+), the process starts and begins to create the temporary directory, but after a while it crashes
I tried with different files on several computers, it always crash after a while if there are too many files in the vpc. Sometimes it goes up to 100 but then it stucks creating the final tif (probably because it don't use bigtiff and is over the 4Gb limit of the regular tiffs).
The text was updated successfully, but these errors were encountered: