Shapefile performance migrating from 0.9 to 1.1 -> 10 times slower .. why?

Topics: SharpMap v0.9 / v1.x
May 5, 2014 at 3:26 PM
Hi

I am migrating a project from 0.9 to 1.1 and have noticed a significant performance drop when running:

``` Map map = new Map(pictureBox1.Size);
VectorLayer layer = new VectorLayer("test");
layer.DataSource = new ShapeFile(@"C:\test.shp", true);
layer.DataSource.Open();

layer.Style.Fill = new SolidBrush(Color.Gainsboro);
layer.Style.EnableOutline = true;
layer.Style.Outline = new Pen(Color.Black);

map.Layers.Add(layer);
map.ZoomToExtents();
pictureBox1.Image = map.GetMap();
` The shape file I am using in above example is around 2.5MB with a no significant dbf file.

Running above code takes 500ms in 0.9 and 5000ms in 1.1

Why is this taking so much longer?

Thanks in advance
Editor
May 6, 2014 at 8:42 AM
I presume it has a file based index? and that it is upto date?
Coordinator
May 6, 2014 at 9:07 AM
Edited May 6, 2014 at 2:10 PM
5 seconds for rendering a bunch of polygons? The only thing I could imagine is that you do have some overhead
  • creating the shx index of the shapefile
  • creating the sidx index of the shapefile
  • for loading NTS,
Do repeated calls to GetMap() take the same amount of time?
May 6, 2014 at 11:44 AM
Robert_Smart: Good guess. I just recreated the index in ArcMAP for the shape file, but it does not improve the performance.

FObermaier: The index should be ok now. Repeated calls to GetMap gives a performance of around 3-400ms in 1.1 where it in 0.9 does not seem to be faster when repeating the call.

So it seems to be something in the initialisation? ... what could be the issue?
Coordinator
May 6, 2014 at 11:46 AM
The spatial indexes from ArcMAP is not used by sharpmap currently since they are not publicly documented. Instead sharpmap creates it's own index (.sidx) on the first call to the shapefile.. This is why the first call takes longer.
May 6, 2014 at 11:55 AM
Ah, ok, yes. Well, I wiped the files so that I only have the .dbf, .shp and .shx file. I can then see that when running the code above that the .shp.sidx is being generated. So I believe SharpMap is successfully creating the index file. Performance remains the same though. Any ideas?
May 6, 2014 at 12:38 PM
Time is lost in ShapeFile.cs
        private Collection<IGeometry> GetGeometriesInViewWithoutFilter(Collection<uint> oids)
        {
            var result = new Collection<IGeometry>();
            using (Stream s = OpenShapefileStream())
            {
                using (BinaryReader br = new BinaryReader(s))
                {
                    using (DbaseReader DbaseFile = OpenDbfStream())
                    {
                        foreach (var oid in oids)
                        {
                            result.Add(GetGeometryByID(oid, br, DbaseFile));
                        }

                        DbaseFile.Close();
                    }
                    br.Close();
                }
                s.Close();
            }

            CleanInternalCache(oids);
            return result;
        }
in the loop

Why is this taking so much longer in 1.1?
May 6, 2014 at 12:43 PM
Above it loops over app oids and does a GetGeometryByID. In there it does a GetFeature which effectively ends up in the shape file being opened and read for every oid. Can I get a second opinion on that perception?
Coordinator
May 6, 2014 at 3:09 PM
GetGeometryById only calls GetFeature if you set UseMemoryCache to true.
Otherwise ReadGeometry is called.
May 7, 2014 at 7:21 AM
Ah, yes ... fantastic.
Getting the same performance as previously now ... which is actually quite good compared to other GIS systems.
Thanks a lot for the swift help :-)