SVG file is enormous

[SteepAtticStairs]

I have been using this library with go-nexrad, and I am able to generate PNG and SVG files. However, the SVG files that are generated are enormous, with a less than 10MB PNG file becoming a 28MB SVG file with the same options. Here is the code:

func render(out string, radials []*archive2.Message31, label string) {

	width := float64(imageSize)
	height := float64(imageSize)

	SVGcanvas := draw2dsvg.NewSvg()
	SVGcanvas.Width = strconv.Itoa(int(width)) + "px"
	SVGcanvas.Height = strconv.Itoa(int(width)) + "px"
	//fmt.Println(canvas.Width)
	//fmt.Println(canvas.Height)
	//draw.Draw(canvas, canvas.Bounds(), image.Black, image.ZP, draw.Src)

	SVGgc := draw2dsvg.NewGraphicContext(SVGcanvas)

	xc := width / 2
	yc := height / 2
	pxPerKm := width / 2 / 460
	firstGatePx := float64(radials[0].ReflectivityData.DataMomentRange) / 1000 * pxPerKm
	gateIntervalKm := float64(radials[0].ReflectivityData.DataMomentRangeSampleInterval) / 1000
	gateWidthPx := gateIntervalKm * pxPerKm

	t := time.Now()
	log.Println("rendering radials")
	// valueDist := map[float32]int{}

	for _, radial := range radials {
		// round to the nearest rounded azimuth for the given resolution.
		// ex: for radial 20.5432, round to 20.5
		azimuthAngle := float64(radial.Header.AzimuthAngle) - 90
		if azimuthAngle < 0 {
			azimuthAngle = 360.0 + azimuthAngle
		}
		azimuthSpacing := radial.Header.AzimuthResolutionSpacing()
		azimuth := math.Floor(azimuthAngle)
		if math.Floor(azimuthAngle+azimuthSpacing) > azimuth {
			azimuth += azimuthSpacing
		}
		startAngle := azimuth * (math.Pi / 180.0)      /* angles are specified */
		endAngle := azimuthSpacing * (math.Pi / 180.0) /* clockwise in radians           */

		// start drawing gates from the start of the first gate
		distanceX, distanceY := firstGatePx, firstGatePx
		SVGgc.SetLineWidth(gateWidthPx + 1)
		SVGgc.SetLineCap(draw2d.ButtCap)

		var gates []float32
		switch product {
		case "vel":
			gates = radial.VelocityData.ScaledData()
		case "sw":
			gates = radial.SwData.ScaledData()
		case "rho":
			gates = radial.RhoData.ScaledData()
		default:
			gates = radial.ReflectivityData.ScaledData()
		}

		numGates := len(gates)
		for i, v := range gates {
			if v != archive2.MomentDataBelowThreshold {

				//fmt.Println(gateWidthPx)
				if i == 0 {
					SVGgc.SetLineWidth(0)
				} else if i > 0 {
					SVGgc.SetLineWidth(gateWidthPx + 1)
				}

				// valueDist[v] += 1

				SVGgc.MoveTo(xc+math.Cos(startAngle)*distanceX, yc+math.Sin(startAngle)*distanceY)

				// make the gates connect visually by extending arcs so there is no space between adjacent gates.
				if i == 0 {
					SVGgc.ArcTo(xc, yc, distanceX, distanceY, startAngle-.001, endAngle+.001)
				} else if i == numGates-1 {
					SVGgc.ArcTo(xc, yc, distanceX, distanceY, startAngle, endAngle)
				} else {
					SVGgc.ArcTo(xc, yc, distanceX, distanceY, startAngle, endAngle+.001)
				}

				SVGgc.SetStrokeColor(colorSchemes[product][colorScheme](v))
				SVGgc.Stroke()
			}

			distanceX += gateWidthPx
			distanceY += gateWidthPx
			azimuth += radial.Header.AzimuthResolutionSpacing()
		}
	}

	// Save to file
	draw2dsvg.SaveToSvgFile(out, SVGcanvas)
	fmt.Println("Finished in", time.Since(t))
}

The full file can be found in my fork of the project here.

The reason I think the SVG is so large is because it is generating the file very inefficiently, possibly by trying to render every pixel instead of just a start and end point. I have tried setting the DPI with SVGgc.setDPI(), but that hasn't worked.

If you have any idea about why the file is so large, or any idea of how to fix it, I would greatly appreciate your input. Hopefully you won't have to go through the entire go-nexrad project to understand this, I have included the code block that I am almost certain is causing the issue, and is the part that uses your library.

If you would like a screen recording of me generating the file and showing the file size with both PNG and SVG, please let me know, if you have difficulty building the project and replicating the issue yourself, if that is needed.

[llgcode]

Hi SteepAtticStairs,

I guess you need to reinitialize the path every time you have drawn something, so that the path don't grow on each loop.

Try to do a SVGgc.BeginPath() and PNGgc.BeginPath() at the start of the loop.

You could also implement a render function that take draw2d.GraphicContext as parameter so that you can pass a SVGgc or PNGgc and factorize some code.

Can you please attach your SVG to better understand the issue ?

regards

[SteepAtticStairs]

radar.svg.gz
Here is the radar.svg file. I gzipped it because the original file was 95.8 MB, but the gzip reduced it to 14.3 MB. This seems like it would be a solution, to just gzip the svg, but then the user would have to unzip it and couldn't render the resulting large svg file.

I tried to do SVGgc.BeginPath() on the line right after I declared SVGgc, but didn't decrease the size. The svg file I have attached resulted from code without SVGgc.BeginPath().

I have also modified the code in my original comment to remove PNGgc, because that works fine. I am able to greatly control the size of a png file by setting the size of the canvas. Examples of this are down below.

1024 x 1024 508 KB

2048 x 2048 1.7 MB

4092 x 4092 5.3 MB

None of these above images have been compressed or processed at all, they are the raw output from draw2dimg.SaveToPngFile(out, PNGcanvas). To see the difference between the images, I would download them and zoom in.

[llgcode]

Thanks,
I've open the svg file. I see a lot of redundant information that could be factorize. For now, the svg generator do not factorize by itself.
I guess one way to factorize is to stroke only when color and lineWidth change. Be careful to not reinitialize the path, I was wrong about the BeginPath in earlier answer, the path is renitialized everytime you draw.
So you can factorize the stroke attribute written into a group. May be you can regroup "draw" that have the same stroke properties.

In our side we could factorize things by using css classes, default values or not using group everytime we draw.

What do you think @drahoslove ?

I see in the SVG path MoveTo and LineTo on the same point, and I couldn't figure out where does this LineTo comes from. It seems that the LineTo is not relevant here. What do you think?

[drahoslove]

@llgcode
I agree with your suggestions.
I hadn't really had an optimization for large files in my mind when I was implementing it.

A simple improvement would be to check whether the group attrs changed and reuse the last group if possible, instead of creating a new one every time.

The default values sounds good as well. We could apply values from first calls of SetStrokeColor, SetFillColor, SetLineWidth, SetLineCap etc. to SVG element itself. And later only create a group only if current values differ from those of the SVG tag. - If that is what you meant by the default values.

The css classes would be nice too, but a bit harder to implement.

I looked into the code and I have no idea where the LineTo comes from either.

@SteepAtticStairs
It looks like you are rendering a lot of transparent pieces (with stroke="rgba(0,0,0,0)") - you could detect those and not render them.
The images are very complex. I'm on the edge about whether svg format is more suitable for them compared to png.
Even if we implement some optimization, the result will probably still be larger than the gzipped version.

[llgcode]

Thanks.
Do you want me to implement the optimisation?
Or you want to do it ?

[llgcode]

I found the redundant LineTo

draw2d/path.go

Line 129 in 80aa0a2

p.LineTo(startX, startY)

[drahoslove]

Thanks. Do you want me to implement the optimisation? Or you want to do it ?

Please, feel free to implement it yourself.
I can review your code, once you are done, if you want me to.

I found the redundant LineTo

Good job!

[SteepAtticStairs]

I have pushed a commit to my project:

5f8a6d0

that prevents transparent colors (e.g. rgba() values that end in 0 which are transparent) from rendering, and I saw a drastic size differential, around 95 MB got reduced to around 60 MB.

I am also gzipping the resulting svg, because this does also reduce the size by an incredible amount.

It looks like you are rendering a lot of transparent pieces (with stroke="rgba(0,0,0,0)") - you could detect those and not render them.

@drahoslove FYI.