v/oct tracking precision?

[clak]

I am curious as to the precision of note values the NerdSeq is capable of. I have a decent meter with good precision and have calibrated my Seq according to the manual.

The 0.000v and -5.000v offsets are exact (no half-digit), the CV/Mod outputs are all exactly 0.000v and 10.00v (I chose the value balancing between 9.999v and 10.00v).

Unfortunately I'm having problems with tuning. I'm using a Tides (2018) as an oscillator, I am running that through a tuner and checking the CV voltage with my meter. On CV1 I have set up from C-0 to C-6 and trigger each one:

C-1 = 0.994v (7.2  cents flat) 
C-2 = 1.998v
C-3 = 3.001v
C-4 = 4.006v (7.2  cents sharp)
C-5 = 5.008v (9.6 cents sharp)
C-6=  6.014v (16.8 cents sharp)

So from C-1 to C-6 there is a difference of 24 cents or 1/4 of a semitone. I admit my meter and math may be off.

I calibrated my non-pro KeyStep with the same meter and was able to get only 5mv difference between C1-C6, 6 cents total and stable, no sharps or flats through the octaves (C1 = 1.000v, C2=2.001v, ..., C6=6.005v). The difference between the KeyStep and NerdSeq is noticeable to my tuner and ear.

I tested all the CV outputs and some of the Mod outputs and the results are all over the place. CV5 out is the most accurate of my outputs across the octaves with only a cent or two variation.

Am I doing something wrong? What is the theoretical precision of the NerdSeq and component tolerances? I can dial in precise voltages using the Mod outputs and hex values, any reason the CV v/oct isn't as precise?

EDIT: I was mistaken, CV5 output isn't much more accurate than the rest, a few consecutive octaves are very close though. Still has 12 cent swings between octaves. I'm guessing the 12 bit nature of the NerdSeq is to blame? I hope I'm wrong.

[XORadmin]

Hey, the NerdSEQ resolution is indeed 12 Bit and this would calculate down to 2,4 millivolt precision. Rounding that up together with the calibration (which fixes the tolerances) you should end up in something like 3 millivolt precision worst case.
The conversion runs linear, so they should be not more than 3 millivolt difference (+-) for each 'position' and that is not audible.

So there is either an issue with your tides (low input resistance which forces the voltage to drop?), measuring or indeed then with your Nerdseq unit.
I can't analyse this from here, but it should normally be correct. I will contact you directly so we can do some tests to find out.

[clak]

Hi, thanks for the quick response. I am ready to do some testing whenever you have the time.

If I'm understanding correctly, no note values should be more than +/- 3mv off, so I shouldn't be seeing C6 as 6.014v etc? Or, for instance, C1 on CV6 gives me 0.991mv.

I have done some more testing today, using 3 different digital oscillators, all with 100K input impedance, with a base tuning at C1 (1.000V). All oscillators track identical when using the same channel and notes from the NerdSeq. I then tested them with a calibrated CV keyboard, all tracked the same and stayed in tune much more than the NerdSeq output as mentioned previously. So the oscillators are tracking correctly (I recalibrated them to make sure) and my CV keyboard is outputting precise voltages.

As far as I can tell the NerdSeq is the culprit as without it I have no tracking issues. I hope it can be calibrated to within +/- 3mv, I have been massaging the calibration values for a channel and jumping back and forth between the calibration and pattern screen to test. While I can influence the tuning that way it isn't fixing the issue.

I would agree that a few mv/cents in either direction isn't audible in a pitch sense to most people, especially non-musicians. I start hearing pitch differences around 12 cents (I'm a pianist which may be why), but most people can hear ~25 cents difference. If you are dealing with a single note/oscillator this isn't such a big deal, as soon as you add another oscillator and it's off only a few cents from the main oscillator you notice instantly both in perceived pitch and in the rhythmic pulsing, especially if the same note is played on both, but even if they are an octave apart.

The biggest issue is that no two CV outputs are the same. If I play a C2 on CV5 I get 2.003v if I play a C2 on CV6 I get 1.990. Even the same CV output can vary ~20mv between different notes/octaves.

[mvdirty]

I noticed this to a minor degree back when first setting up my NerdSEQ and expanders, and followed this thread as it was posted. Today, though, during some setup and metering and patching I observed it occurring strongly enough on one of my CV16 outputs that it would be out by a semitone if I went beyond setup and metering via the Sinfonion (which is my main pitch destination and which I used today to check and calibrate all of the outputs on two NerdSEQs with three CV 16 expanders.)

Perhaps interestingly, I’m not observing just an offset or a linear over-scaling or under-scaling of the values but rather a non-linear response, at least on this particular CV16 output. Other outputs fare better, but not all have been checked in as much detail. Most have been checked just in terms of min, max, and midpoint, with and without the -5V switched offset.

I suspect NerdSEQ could benefit from a calibration process utilizing more (hopefully far more) than two points, not unlike how is the case for some VCO calibration modules ala CalTrans, even if just for the CV16 expanders if the main module couldn’t spare the memory for it, but even better if it could.

How about it, Thomas, could we at some point have eleven-point calibration for every output (and perhaps the inputs too?) A value for every volt?

[XORadmin]

I noticed this to a minor degree back when first setting up my NerdSEQ and expanders, and followed this thread as it was posted. Today, though, during some setup and metering and patching I observed it occurring strongly enough on one of my CV16 outputs that it would be out by a semitone if I went beyond setup and metering via the Sinfonion (which is my main pitch destination and which I used today to check and calibrate all of the outputs on two NerdSEQs with three CV 16 expanders.)

Perhaps interestingly, I’m not observing just an offset or a linear over-scaling or under-scaling of the values but rather a non-linear response, at least on this particular CV16 output. Other outputs fare better, but not all have been checked in as much detail. Most have been checked just in terms of min, max, and midpoint, with and without the -5V switched offset.

I suspect NerdSEQ could benefit from a calibration process utilizing more (hopefully far more) than two points, not unlike how is the case for some VCO calibration modules ala CalTrans, even if just for the CV16 expanders if the main module couldn’t spare the memory for it, but even better if it could.

How about it, Thomas, could we at some point have eleven-point calibration for every output (and perhaps the inputs too?) A value for every volt?

I think this is very strange. I usually don't have any issues with that. Also, most of the outputs share the same DAC on multiple outputs. And these have a linear response by specification with a error rate of +- 1 LSB which is like 1/34 of a semitone over the full range.

But basically i think even a per note calibration is possible. But that would be a long job for each output to calibrate them. (Just imagine 16 outputs, each 120 notes....).

[mvdirty]

Preface: All outputs I’m about to describe are within a millivolt or two at their 0V and 10V extents.

I should clarify that most of the outputs where I’ve noticed larger issues can be off by up to a half semitone at some point(s) in their range. These could be off by a semitone once processed downstream by certain modules, due to their quantization of the values, and hence my mention of semitone in my previous reply.

I do have at least one CV16 output where 5V is being output as 4.985 which, as you might imagine, makes that output unusable for pitch whether quantized downstream or not.

(Switching outputs to bipolar definitely throws some off more than others and recalibration is clearly necessary after throwing any switch. Most of my bipolar measurements don’t account for that yet so I’ll have to redo them at some point. As such, this message only covers unipolar output issues.)

I could work up a more detailed spreadsheet in relation to some or all of these outputs but, with two NerdSEQs and three CV16 modules, that would be quite a job all at once. What I’ll perhaps do instead is set up that spreadsheet today but just test and fill in values as I go to use each output. The timing is relatively good as I’m currently building up a big new patch heading into the new year.

As for more detailed calibration possibilities, I would hope that even just a few more calibration points per output would be sufficient but one per volt might help balance detail against calibration simplicity. That said, perhaps this could be both more accurate and simpler still...

I imagine something roughly like the following: Manually calibrate a single output at much greater detail, at least volts, perhaps even tenths of volts. Then use that output to perform a very detailed automatic calibration of the four CV inputs. Then use those CV inputs for automatic loopback calibration of the other outputs, each at rather great detail. (Optimally with the ability to dive down and manually tweak each value in the calibration results if needed for some reason.)

---

On a related thought: Is there enough horsepower in NerdSEQ and/or CV16 to turn this into a (reasonably low-order) polynomial curve-fitting problem? It’d produce a better result, and likely with far fewer values to adjust, but I would totally understand if there’s only enough horsepower for linear interpolation rather than calculation from polynomials.

[XORadmin]

Preface: All outputs I’m about to describe are within a millivolt or two at their 0V and 10V extents.

I should clarify that most of the outputs where I’ve noticed larger issues can be off by up to a half semitone at some point(s) in their range. These could be off by a semitone once processed downstream by certain modules, due to their quantization of the values, and hence my mention of semitone in my previous reply.

I do have at least one CV16 output where 5V is being output as 4.985 which, as you might imagine, makes that output unusable for pitch whether quantized downstream or not.

(Switching outputs to bipolar definitely throws some off more than others and recalibration is clearly necessary after throwing any switch. Most of my bipolar measurements don’t account for that yet so I’ll have to redo them at some point. As such, this message only covers unipolar output issues.)

I could work up a more detailed spreadsheet in relation to some or all of these outputs but, with two NerdSEQs and three CV16 modules, that would be quite a job all at once. What I’ll perhaps do instead is set up that spreadsheet today but just test and fill in values as I go to use each output. The timing is relatively good as I’m currently building up a big new patch heading into the new year.

As for more detailed calibration possibilities, I would hope that even just a few more calibration points per output would be sufficient but one per volt might help balance detail against calibration simplicity. That said, perhaps this could be both more accurate and simpler still...

I imagine something roughly like the following: Manually calibrate a single output at much greater detail, at least volts, perhaps even tenths of volts. Then use that output to perform a very detailed automatic calibration of the four CV inputs. Then use those CV inputs for automatic loopback calibration of the other outputs, each at rather great detail. (Optimally with the ability to dive down and manually tweak each value in the calibration results if needed for some reason.)

---

On a related thought: Is there enough horsepower in NerdSEQ and/or CV16 to turn this into a (reasonably low-order) polynomial curve-fitting problem? It’d produce a better result, and likely with far fewer values to adjust, but I would totally understand if there’s only enough horsepower for linear interpolation rather than calculation from polynomials.

I don't really think it is necessary to have any extra adjustment points as again the response of the DAC is linear by specifications. If it is not the case with you then you got most likely a hardware problem and should contact me directly. Any additional adjustment points would be a workaround only I think (thinking twice about it)

As for the bipolar vs. unipolar. The calibration is being done with the unipolar settings as these are the only ones that matter for music notes. Bipolar setup is for modulations.

[clak]

I don't really think it is necessary to have any extra adjustment points as again the response of the DAC is linear by specifications. If it is not the case with you then you got most likely a hardware problem and should contact me directly. Any additional adjustment points would be a workaround only I think (thinking twice about it)

I think this is disingenuous on your part. No hardware is perfect, there is a margin of error in all hardware that needs to be accounted for somehow, i.e. software calibration. It is not a workaround if you don't verify the response of every DAC that goes into your product.

The NerdSeq is already hindered by the use of a 12bit DAC for pitch duties in my opinion. 12bit alone is barely adequate for standard chromatic tuning, much less more exotic ones. Being 12bit and only using 2 calibration points is only asking for problems.

I have a Mutable Yarns which has performed flawlessly for me, it is a very precise piece of hardware. From the spec sheet for Yarns:

High-end 16-bit DAC and voltage references.
12-point calibration table, adjusted in factory.

I trust Émilie when she uses the term High-end. Not only is the DAC 16bit and "High-end", she still uses a 12 point calibration table. I wouldn't call it a workaround, I would say it is the right way to do it.

Another Mutable example is Marbles:

14-bit DAC with accurate software calibration - error below 1mV.

I don't have one to verify but I imagine similar calibration tricks are used in this module in order to make this claim.

Have you ever verified the linearity of the DACs you use?

[XORadmin]

I don't really think it is necessary to have any extra adjustment points as again the response of the DAC is linear by specifications. If it is not the case with you then you got most likely a hardware problem and should contact me directly. Any additional adjustment points would be a workaround only I think (thinking twice about it)

I think this is disingenuous on your part. No hardware is perfect, there is a margin of error in all hardware that needs to be accounted for somehow, i.e. software calibration. It is not a workaround if you don't verify the response of every DAC that goes into your product.

The NerdSeq is already hindered by the use of a 12bit DAC for pitch duties in my opinion. 12bit alone is barely adequate for standard chromatic tuning, much less more exotic ones. Being 12bit and only using 2 calibration points is only asking for problems.

I have a Mutable Yarns which has performed flawlessly for me, it is a very precise piece of hardware. From the spec sheet for Yarns:

High-end 16-bit DAC and voltage references.
12-point calibration table, adjusted in factory.

I trust Émilie when she uses the term High-end. Not only is the DAC 16bit and "High-end", she still uses a 12 point calibration table. I wouldn't call it a workaround, I would say it is the right way to do it.

Another Mutable example is Marbles:

14-bit DAC with accurate software calibration - error below 1mV.

I don't have one to verify but I imagine similar calibration tricks are used in this module in order to make this claim.

Have you ever verified the linearity of the DACs you use?

I did several times over the years. And a 12 bit precision is not uncommon. 16 bit precision is of course better but this was a design decision many years ago and it has never been an issue.

[mvdirty]

Ugh, this is somewhat embarrassing but I think my brain has been going blurry with everything I’ve been testing and I may have misspoken in an earlier message, I suspect due to the amount of number systems I’ve been translating on the fly, the seven-year-old constantly talking to me while I’m trying to test things,  etc. I bet someone can spot the main error in one of my messages above.  It’s not small. 

I’m going to refrain from drawing further conclusions or even providing much more commentary (beyond the below) until I can slow way down and do detailed testing that includes careful automated conversion of all the various values involved. Spreadsheet time here I come. Ugh. 72 outputs to test, possibly semitone by semitone if I need to zoom in on a problem output but more generally octave by octave for a start.

It’ll be an interesting take on Jamuary but I need to know the state of these outputs before the big patch, far more than I need a daily jam.

I guess the short version for now is that:

• On unipolar outputs I can confirm at least some non-linearity on some, though it may end up falling within design tolerances. I’ll produce more detailed testing results.
  
• On bipolar outputs I can at least confirm that very few offset by exactly -5V. Many are off by 10 to 30 millivolts and some are off by 30 to 50. This too may be considered within tolerance. I have yet, however, to properly confirm any additional non-linearity but I’ll include that in my further testing.
  

I don't really think it is necessary to have any extra adjustment points as again the response of the DAC is linear by specifications. If it is not the case with you then you got most likely a hardware problem and should contact me directly.

Hopefully I won’t have to, but I’ll reach out if more detailed testing of everything shows anything odd.

As for the bipolar vs. unipolar. The calibration is being done with the unipolar settings as these are the only ones that matter for music notes. Bipolar setup is for modulations.

My use case is perhaps uncommon, but just to register it for the record: I use a few bipolar outputs for what are effectively pitch offsets.

Most of these are summed with at least one other offset, via precision adders. Some are then further summed with actual pitches. The results are then interpreted by what you can think of as semitone quantization logic. It’s a long story.

Regarding clak’s point about DAC bit depth and multi-point calibration: We can debate the finer points of necessity but he’s not wrong in that there are software approaches by which better output can be achieved, and perhaps hardware improvements that should happen in the future in order to provide advancement (Portable, perhaps?)

I’m going quiet on this thread for now. I really shouldn’t say much more without proper detailed test results rather than my current spot checks.