How fast are you able to upload a file?

Post Reply
bob64
Printmaster!
Posts: 89
Joined: Sun Jul 20, 2014 8:45 pm

How fast are you able to upload a file?

Post by bob64 »

Trying to figure if I need a faster microsd card or something because I'm getting ~250-300kbps and it's really painful when uploading 50mb gcode files.

Edit: I'm connected via gigabit switch through the web interface.
Last edited by bob64 on Mon Mar 28, 2016 12:36 pm, edited 1 time in total.
Rostock Max V2 with mods:
E3d v6 with 713maker mount & e3d titan extruder
Tricklaser CF arms, heat spreader, tricktrucks, fly-n-strude, metal carriage & effector
Duet v.8.5.0
24v MeanWell 750w PS & Cydom D1D40 SSR in parallel with stock 12v PS

geneb
ULTIMATE 3D JEDI
Posts: 5161
Joined: Mon Oct 15, 2012 12:47 pm
Location: Graham, WA
Contact:

Re: How fast are you able to upload a file?

Post by geneb »

That's basically the serial rate between your pc and the printer. Get yourself an SD card reader/writer and just copy the file to it.

g.
Delta Power!
Defeat the Cartesian Agenda!
http://www.f15sim.com - 80-0007, The only one of its kind.
http://geneb.simpits.org - Technical and Simulator Projects

User avatar
mhackney
ULTIMATE 3D JEDI
Posts: 5412
Joined: Mon Mar 26, 2012 4:15 pm
Location: MA, USA
Contact:

Re: How fast are you able to upload a file?

Post by mhackney »

According to Christian, with a fast card uploads on wired ethernet of 400KBPS are typical and wireless/bridge (WR701N) around 280KBPS (which is exactly what I'm getting). Those are big files, mine are typically 5MB.

Sublime Layers - my blog on Musings and Experiments in 3D Printing Technology and Art

Start Here:
A Strategy for Successful (and Great) Prints

Strategies for Resolving Print Artifacts

The Eclectic Angler

dc42
Printmaster!
Posts: 453
Joined: Mon Mar 07, 2016 10:17 am

Re: How fast are you able to upload a file?

Post by dc42 »

I made some improvements to the SD card write code in my 1.12 release (now superceded by 1.12a) and I am now getting 900kbytes/sec on one of my printers. The other printer is delivering about 500kbytes/sec, which although slower is a big improvement on the speed it was giving before. The SD card you use can still make a big difference.

A tip: if you want to upload and print a large file, set the bed temperature before you start the upload. The file will probably have finished uploading by the time the bed is up to temperature.

bob64
Printmaster!
Posts: 89
Joined: Sun Jul 20, 2014 8:45 pm

Re: How fast are you able to upload a file?

Post by bob64 »

Strange. The web interface still shows around 200kbps-300kbps with 1.12a. I'm using Class 10/ XC 1 card.
Rostock Max V2 with mods:
E3d v6 with 713maker mount & e3d titan extruder
Tricklaser CF arms, heat spreader, tricktrucks, fly-n-strude, metal carriage & effector
Duet v.8.5.0
24v MeanWell 750w PS & Cydom D1D40 SSR in parallel with stock 12v PS

dc42
Printmaster!
Posts: 453
Joined: Mon Mar 07, 2016 10:17 am

Re: How fast are you able to upload a file?

Post by dc42 »

I did some measurements on file upload speeds using various SD cards and two firmware versions. Here are my results:

Code: Select all

Card                   Duet 0.8.5    Duet 0.8.5    Duet NG prototype
                       1.10+4-dc42   1.12a-dc42    1.12a
2Gb unbranded            93           235           577
2Gb Samsung             146           348           786
4Gb Kingston class 10   227           550           856
4Gb Sandisk class 4     430           783           840
16Gb Sandisk class 4    373           986           895

These figures demonstrate how much the upload speed depends on the SD card, and the effect of the improvements I made in release 1.12. There was some variability, so all speeds listed here are best of three uploading a 10Mb file.

The 4Gb Sandisk card came from T3P3 and I believe it is the card they are currently shipping with Duet 0.8.5 boards. The 16Gb Sandisk card is this one https://www.amazon.co.uk/gp/product/B00 ... ge_o06_s00 and the 4Gb Kingston card is this one https://www.amazon.co.uk/gp/product/B00 ... ge_o07_s00.

Anyone who is getting better than 1Mb/sec on a 10Mb or larger file upload, please let us know!

Eric
Printmaster!
Posts: 726
Joined: Sat Aug 18, 2012 4:09 am
Location: Chula Vista, CA

Re: How fast are you able to upload a file?

Post by Eric »

dc42 wrote:I did some measurements on file upload speeds using various SD cards and two firmware versions. Here are my results:

Code: Select all

Card                   Duet 0.8.5    Duet 0.8.5    Duet NG prototype
                       1.10+4-dc42   1.12a-dc42    1.12a
2Gb unbranded            93           235           577
2Gb Samsung             146           348           786
4Gb Kingston class 10   227           550           856
4Gb Sandisk class 4     430           783           840
16Gb Sandisk class 4    373           986           895

These figures demonstrate how much the upload speed depends on the SD card, and the effect of the improvements I made in release 1.12. There was some variability, so all speeds listed here are best of three uploading a 10Mb file.

The 4Gb Sandisk card came from T3P3 and I believe it is the card they are currently shipping with Duet 0.8.5 boards. The 16Gb Sandisk card is this one https://www.amazon.co.uk/gp/product/B00 ... ge_o06_s00 and the 4Gb Kingston card is this one https://www.amazon.co.uk/gp/product/B00 ... ge_o07_s00.

Anyone who is getting better than 1Mb/sec on a 10Mb or larger file upload, please let us know!


What those numbers tell me is that the current bottleneck isn't the hardware, it's the software. You already know this, I think, since your improvements helped. If it was any part of the hardware other than the SD card itself, you'd be getting similar numbers for all cards.

Still, you're nowhere near the published write specs of those SD cards, and they all can be written much faster by PC software. That leads me to think that there is significant room for improvement, probably in error-checking or buffer handling, since you are seeing differences between cards. Unless you're depending on someones library, in which case you may be hitting the limits of that library implementation.

dc42
Printmaster!
Posts: 453
Joined: Mon Mar 07, 2016 10:17 am

Re: How fast are you able to upload a file?

Post by dc42 »

Eric, as I explained before (not sure whether in this thread or another one) these cards are optimised for writing large blocks of memory, as happens in digital cameras. The headline speeds you see advertised are only achievable when writing large blocks. Unfortunately we don't have megabytes of RAM buffer available on the Duet, so we are forced to write much smaller blocks than the card is optimised for. When the card sees the end of a block, it assumes that you won't be writing another one for a little while, and goes off to do internal wear-levelling and other reorganisation. Then, when you want to write the next block, it won't accept a new block until it has finished doing that. Different cards use different algorithms for wear levelling and block management, so the amount of time you have to wait before the card can accept another block varies a lot between cards. The bottleneck is definitely the latency of the card in accepting new blocks.

If we can reduce the RAM usage of other parts of the firmware, we may be able to increase the block size some more, which should increase the speed. We know that the network stack can deliver file upload data at 3Mbytes/sec, and the blocks are transferred to the card at around 20Mbytes/second. So in theory it should be possible to achieve at least 2Mbytes/sec with a large enough buffer, and perhaps more with improvements to the network stack.

In principle it might be possible to stream data directly from the network buffers to the card without using a write buffer. However, that would require a rewrite of a large part of the file system code.

Post Reply

Return to “Duet”