Merge grid update by sindizzy · Pull Request #1482 · DotSpatial/DotSpatial · GitHub
Skip to content

Merge grid update#1482

Draft
sindizzy wants to merge 12 commits into
DotSpatial:masterfrom
sindizzy:Perez-MergeGridUpdate
Draft

Merge grid update#1482
sindizzy wants to merge 12 commits into
DotSpatial:masterfrom
sindizzy:Perez-MergeGridUpdate

Conversation

@sindizzy

@sindizzy sindizzy commented Dec 26, 2022

Copy link
Copy Markdown
Contributor

Fixes #1443. This PR suggests a method to intelligently determine the best data type for the output raster. Production is hard coded to generate a raster of type int so I introduced some crude methods to determine the best data type fit. However, the routines that determine the best data type are only called once per operation so it's not too expensive. I am attaching data type logic testing and will follow up with actual grid testing.

Checklist

  • I have included examples or tests
  • I have updated the change log
  • I am listed in the CONTRIBUTORS file
  • I have cleaned up the commit history (use rebase and squash)

Changes proposed in this pull request:

Due to the issue #1443 I started to look at the routine thinking it was an easy fix. It was not an easy fix. The aim of the routine DotSpatial.Tools.MergeGrids.Execute is to take two grids and merge them. Easy enough, but due to a myriad of raster types, the current routine had to be adjusted.

  • Currently the output raster is always of type int. Obviously, rasters can be floating point so right off the bat this has to be changed.
  • This PR attempts to classify the data types that are supported by the output raster file format and then determine which of those types can hold the values of both the input rasters.
  • I believe this is a first step to making a more robust Merge Grids routine, as in the future we can allow the user to determine the output type OR auto-select it for them.
  • Note that this may be overkill because I am assuming all the raster functionality goes through GDAL and it would probably throw an exception if a grid format did not support a data type as requested by the user. Maybe this PR code is needed, maybe not. I am open to discussion.
  • The other subject in issue bug report in Dotspatial.Tool.MergeGrids  #1443 was stated as "I get an exception from “_output.Value[i, j] = output.NoDataValue” which could happen if the NoDataValue is set to a non-finite value and the output raster is of type int. If the resultant cell value is NaN then it follows that we cannot store that in an int raster.

sindizzy and others added 6 commits November 30, 2022 19:32
added logic for closest data type
Some minimal changes to comments.
Major changes to the logic that determines the best data type for the output raster.
Some test files that show the output of the data type logic for various cases.
small updates
@jany-tenaj

Copy link
Copy Markdown
Contributor

@sindizzy

sindizzy commented Jan 5, 2023

Copy link
Copy Markdown
Contributor Author

@jany-tenaj let me review the IsFloatingPoint routine. I remember I had some issues with it but it did compile. Ill take a look again.

The test.txts files are nothing more than simple tests to see how the logic to determine the best output data type works. For example, if the inputs are DTED and ASCII and the requested output is GRB what would be the best data type for the merge. I just dont have access to all the raster formats and data types.

@jany-tenaj jany-tenaj marked this pull request as draft January 13, 2023 12:07
Added test cases and raster data.
The merge test showed that the new logic wasn't quite working as expected.
Added back the original logic until the new logic can be reviewed.
Small updates
@sindizzy

Copy link
Copy Markdown
Contributor Author

@jany-tenaj ok I have updated IsFloatingPoint so that it now has no warnings.
I added tests for IsFloatingPoint and GetClosestType and the results look very good.
Then I added SimpleMergeGridsTest for some sample grids. It was failing so I named my routine ExecuteNew and re-added the production logic for Execute.

Oddly enough the current production logic also bombs. Maybe I am missing something on this method.
The FileType and DataType do not seem to reflect what's in the input rasters.

fileNameA=C:\Users\DarkPax\Documents\GitHub\DotSpatial\Source\Tests\DotSpatial.Tools.Tests\bin\Data\Grids\TIFF\GTOPO30.tif
gridA: min=137 max=238 cols=39 rows=24
gridA: FileType=Ascii DataType=System.Int16

fileNameB=C:\Users\DarkPax\Documents\GitHub\DotSpatial\Source\Tests\DotSpatial.Tools.Tests\bin\Data\Grids\BIL\GTOPO30.bil
gridB: min=125 max=215 cols=39 rows=24
gridB: FileType=Ascii DataType=System.Int16

fileNameOut=C:\Users\DarkPax\Documents\GitHub\DotSpatial\Source\Tests\DotSpatial.Tools.Tests\bin\Data\Grids\merged.bgd

The size of the file was 558 which didn't match the expected 5586380
Exception thrown: 'System.NullReferenceException' in DotSpatial.Data.dll
An exception of type 'System.NullReferenceException' occurred in DotSpatial.Data.dll but was not handled in user code
Object reference not set to an instance of an object.

@sindizzy

Copy link
Copy Markdown
Contributor Author

Yeah I think something is not right.

var rstTypeA = Raster.GetGridFileType(fileNameA);
Debug.Print("gridA: RasterFileType={0}", rstTypeA);

var p = new GdalRasterProvider();
var gridA = p.Open(fileNameA);
Debug.Print("gridA: FileType={0} DataType={1}", gridA.FileType, gridA.DataType);

This outputs

fileNameA=\Source\Tests\DotSpatial.Tools.Tests\bin\Data\Grids\TIFF\GTOPO30.tif
gridA: RasterFileType=GeoTiff
gridA: FileType=Ascii DataType=System.Int16

It appears that the FileType (and maybe some other properties) are not set with the GdalRasterProvider.Open method. Should I be using Raster.Open instead?

@jany-tenaj

Copy link
Copy Markdown
Contributor

I've got no idea. Never used this tool before

@sindizzy

Copy link
Copy Markdown
Contributor Author

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bug report in Dotspatial.Tool.MergeGrids

2 participants