Tile rendering WMS

Topics: Web Controls
May 26, 2015 at 7:47 AM
Hi Sharpmappers,

I am making a WMS service that is able to handle different types of data. I am using SharpMap 1.0 (the downloaded version).

For Shapefiles the server is working perfectly. The files are reprojected on the fly and projected as a layer in an OpenLayers application.

For the Rasterfiles I have decided to use Gdal Warp to reproject the files into a temporary file (see thread 611767). The layer tiles are rendered from these temporary files. For some unknown reasons some of the tiles are rendered as a black image and others are rendered correctly. After a while (zooming in and out) the problem disappears. I have spent quite some time trying to fix this problem but I still have no idea where the problems comes from.

I suspect that is has something to do with a (GDI) file-lock after it is created. It looks like the file does not except too many request at the same time just after it is created. I have read some Threads about a GDI+ problem, but did not really find a solution yet.

I suppose the problem is somewhere in GdalRasterLayer.cs, but I do not know which part of code is causing it. I was hoping one of you could point me in the right direction, so I could try to fix it. In the case it is too complex to fix I would like to try a workaround by checking if the error occurs and redraw the tile. Because I am not going to check all cells one-by-one I need to find which piece of code is causing the problem.

I am creating the temporary file as follows:
               // creating a reprojected temporary raster file. Reprojecting on the fly is too heavy to compute.
                Gdal.AllRegister();
                Dataset ra;
                Dataset warp;
                // declaring the GDAL driver
                Driver dout;   

                 // determine target format    
                if (Path.GetExtension(path) == ".asc")
                {
                    dout = Gdal.GetDriverByName("AAIGrid");

                }
                else
                {
                    dout = Gdal.GetDriverByName("GTiff");
                }

                // target projection
                string refnew = "PROJCS[\"WGS 84 / Pseudo-Mercator\",GEOGCS[\"WGS 84\",DATU....etc.;

                // open the file
                try
                {
                    ra = Gdal.Open(path, Access.GA_ReadOnly);
                }
                catch
                {
                    WmsException.ThrowWmsException("unable to open file");
                    return "";
                }

                // get current projection
                string refone = ra.GetProjection();

                try
                {
                    // rewarp into new projection
                    warp = Gdal.AutoCreateWarpedVRT(ra, refone, refnew, ResampleAlg.GRA_NearestNeighbour, 0.0);
                }
                catch
                {
                    // if not able to reproject create an image
                    ra.Dispose();
                    WmsException.ThrowWmsException("The projection is not supported try: " + ImageHelper.rasterMap(path));
                    return "";
                }             

                // save temporary image. 
                dout.CreateCopy(url, warp, 0, null, null, "");
                warp.Dispose();
                ra.Dispose();
And rendering the tiles as follows:
        //creating the raster map for the tiles
        public static Map RasterTiles(string path)
        {
            Map map = new Map(new Size(1, 1));

            // The layer is always called "shapefile" since there is only one layer at a time
            GdalRasterLayer rlay = new GdalRasterLayer("shapefile", path);

            rlay.Transparency = 20;
            // adding layer to map
            map.Layers.Add(rlay);

            return map;
        }
Perhaps a little bit off topic but it could be important for people with a related issue:

I also found another problem with GDI on IIS express. I received a general GDI+ error. Apparently this error only occurs under certain circumstances. I had to make the following workaround at the end of Cache.cs in sharpmap.web to let it work (really ugly):
MemoryStream tempStream = new MemoryStream();
            map.Save(tempStream, ImageFormat.Png);
            map.Dispose();
            Image map2 = Image.FromStream(tempStream);

            using (var stream = new MemoryStream())
            {
                map2.Save(stream, ImageFormat.Png);
                HttpContext.Current.Cache.Insert(guid, stream.ToArray(), null, 
                                                Cache.NoAbsoluteExpiration,
                                                 TimeSpan.FromMinutes(minutes));
            }
            map2.Dispose();
I work on a windows XP computer with visual 2010 + IIS express server.

Sorry for the long post and thanks in advance.

Kas
Coordinator
May 27, 2015 at 11:57 AM
You are using the 1.0 release or the 1.1 release? If I'm not mistaken there has been work on cuncurrency issues with GdalRasterLayer. It might be post 1.1 though.
Could you try to compile the solution yourself and see if your issue is resolved?

Regarding the SharpMap.Web issue I have no clue if others experience this, too.
May 29, 2015 at 2:22 PM
Hi FObermaier,

Thanks for your reply. I am using the 1.0 branch version and compiled it myself.

It looks like this version is newer than 1.1?
Coordinator
Jun 1, 2015 at 7:16 AM
If you use a self compiled version you should be all set.
IIRC there are some GDAL data providers that are not thread-safe themselves so there is little we can do to help that within SharpMap.
Can you reproduce black or blank images for certain viewport/size combinations, that would be a start to investigate what happens.
Is there a no data value set? Is it an aerial image or some elevation data that is to be interpreted by some color gradient?
Jun 8, 2015 at 3:22 PM
IIRC there are some GDAL data providers that are not thread-safe themselves so there is little we can do to help that within SharpMap.
I don't know why I didn't try this before. I played around with some different formats and these are my findings:
  • Tiff (my original format, not thread safe) --> the tile rendering problem occurs and dissapears after a while
  • Bitmap (supposed to be thread safe) --> Similar problem. During the first few attempts the extent cannot be read. The first tile is deformed and located at 0,0. Most other tiles are shown without problem
  • GDAL VRT (single threading) --> works but terribly slow, so not suitable for rendering
  • JPEG --> Same result as Bitmap
  • IMG --> Not displayed on the map, I did no further research why
  • ECW (Licence is needed for usage on a server) --> Works without any problems. Performances are also not bad
  • ASCII XYZ --> Not displayed on the map, I did no further research why
Can you reproduce black or blank images for certain viewport/size combinations, that would be a start to investigate what happens.
I'm not quite sure if this is what you mean: zooming the map does give the same results. It is only dependent on the time between the rendering of a tile and the creation of the file. The larger the difference in time the less likely the tile is rendered incorrectly.
Is there a no data value set? Is it an aerial image or some elevation data that is to be interpreted by some color gradient?
There is no no-data value set. It happens with all different kinds of datasets: Tiff, ECW, JPG etc. I did not find a dataset with different results. For the readers being confused by the different datasets: these are the datasets before rewarping.

It seems that the issue is dependent on the data provider as FObermaier suggested. For now, the only data provider without any problems is ECW. I am not sure yet what will happen with the other formats if it is deployed on a server. I fear I will have the same result.