How fast are you able to upload a file?

[bob64]

Trying to figure if I need a faster microsd card or something because I'm getting ~250-300kbps and it's really painful when uploading 50mb gcode files.

Edit: I'm connected via gigabit switch through the web interface.

[geneb]

That's basically the serial rate between your pc and the printer. Get yourself an SD card reader/writer and just copy the file to it.

g.

[mhackney]

According to Christian, with a fast card uploads on wired ethernet of 400KBPS are typical and wireless/bridge (WR701N) around 280KBPS (which is exactly what I'm getting). Those are big files, mine are typically 5MB.

[dc42]

I made some improvements to the SD card write code in my 1.12 release (now superceded by 1.12a) and I am now getting 900kbytes/sec on one of my printers. The other printer is delivering about 500kbytes/sec, which although slower is a big improvement on the speed it was giving before. The SD card you use can still make a big difference.

A tip: if you want to upload and print a large file, set the bed temperature before you start the upload. The file will probably have finished uploading by the time the bed is up to temperature.

[bob64]

Strange. The web interface still shows around 200kbps-300kbps with 1.12a. I'm using Class 10/ XC 1 card.

[dc42]

I did some measurements on file upload speeds using various SD cards and two firmware versions. Here are my results:

Code: Select all

Card                   Duet 0.8.5    Duet 0.8.5    Duet NG prototype
                       1.10+4-dc42   1.12a-dc42    1.12a
2Gb unbranded            93           235           577
2Gb Samsung             146           348           786
4Gb Kingston class 10   227           550           856
4Gb Sandisk class 4     430           783           840
16Gb Sandisk class 4    373           986           895

These figures demonstrate how much the upload speed depends on the SD card, and the effect of the improvements I made in release 1.12. There was some variability, so all speeds listed here are best of three uploading a 10Mb file.

The 4Gb Sandisk card came from T3P3 and I believe it is the card they are currently shipping with Duet 0.8.5 boards. The 16Gb Sandisk card is this one https://www.amazon.co.uk/gp/product/B00 ... ge_o06_s00 and the 4Gb Kingston card is this one https://www.amazon.co.uk/gp/product/B00 ... ge_o07_s00.

Anyone who is getting better than 1Mb/sec on a 10Mb or larger file upload, please let us know!

[Eric]

I did some measurements on file upload speeds using various SD cards and two firmware versions. Here are my results:

Code: Select all

Card                   Duet 0.8.5    Duet 0.8.5    Duet NG prototype
                       1.10+4-dc42   1.12a-dc42    1.12a
2Gb unbranded            93           235           577
2Gb Samsung             146           348           786
4Gb Kingston class 10   227           550           856
4Gb Sandisk class 4     430           783           840
16Gb Sandisk class 4    373           986           895

These figures demonstrate how much the upload speed depends on the SD card, and the effect of the improvements I made in release 1.12. There was some variability, so all speeds listed here are best of three uploading a 10Mb file.

The 4Gb Sandisk card came from T3P3 and I believe it is the card they are currently shipping with Duet 0.8.5 boards. The 16Gb Sandisk card is this one https://www.amazon.co.uk/gp/product/B00 ... ge_o06_s00 and the 4Gb Kingston card is this one https://www.amazon.co.uk/gp/product/B00 ... ge_o07_s00.

Anyone who is getting better than 1Mb/sec on a 10Mb or larger file upload, please let us know!

What those numbers tell me is that the current bottleneck isn't the hardware, it's the software. You already know this, I think, since your improvements helped. If it was any part of the hardware other than the SD card itself, you'd be getting similar numbers for all cards.

Still, you're nowhere near the published write specs of those SD cards, and they all can be written much faster by PC software. That leads me to think that there is significant room for improvement, probably in error-checking or buffer handling, since you are seeing differences between cards. Unless you're depending on someones library, in which case you may be hitting the limits of that library implementation.

[dc42]

Eric, as I explained before (not sure whether in this thread or another one) these cards are optimised for writing large blocks of memory, as happens in digital cameras. The headline speeds you see advertised are only achievable when writing large blocks. Unfortunately we don't have megabytes of RAM buffer available on the Duet, so we are forced to write much smaller blocks than the card is optimised for. When the card sees the end of a block, it assumes that you won't be writing another one for a little while, and goes off to do internal wear-levelling and other reorganisation. Then, when you want to write the next block, it won't accept a new block until it has finished doing that. Different cards use different algorithms for wear levelling and block management, so the amount of time you have to wait before the card can accept another block varies a lot between cards. The bottleneck is definitely the latency of the card in accepting new blocks.

If we can reduce the RAM usage of other parts of the firmware, we may be able to increase the block size some more, which should increase the speed. We know that the network stack can deliver file upload data at 3Mbytes/sec, and the blocks are transferred to the card at around 20Mbytes/second. So in theory it should be possible to achieve at least 2Mbytes/sec with a large enough buffer, and perhaps more with improvements to the network stack.

In principle it might be possible to stream data directly from the network buffers to the card without using a write buffer. However, that would require a rewrite of a large part of the file system code.