Solving a linear system with physical meaning  Printable Version + HP Forums (https://archived.hpcalc.org/museumforum) + Forum: HP Museum Forums (https://archived.hpcalc.org/museumforum/forum1.html) + Forum: Old HP Forum Archives (https://archived.hpcalc.org/museumforum/forum2.html) + Thread: Solving a linear system with physical meaning (/thread131049.html) 
Solving a linear system with physical meaning  Rodger Rosenbaum  01192008 On page 1812 of the HP48G User's Guide, it says: "Be sure to pay attention to the nature of the linear system you are solving because it will influence how you should interpret the result array. In some cases, you will want to test for illconditioning before accepting even an exact solution as a "true" solution." I will give a system that exemplifies a situation where the exact solution should probably not be considered the "true" solution. Imagine that you need to measure an electric current, and to increase accuracy you will make the measurement with 4 nominally identical meters and derive some kind of weighted average of the readings of the 4 meters. These meters are just simple analog panel meters. To get a weighting factor for each meter, you connect the 4 meters in series and pass an accurately known current through all 4 meters at the same time. Pass a current of 1, 2, 3 and 4 amperes through the meters, and note the reading on each meter. Create a matrix of the meter readings and set up a linear sytem like this: A*W = B A is the matrix of meter readings and B is the column matrix of the accurate currents that were passed through the meters in series. W will be the weighting factors to be applied to the meter readings in the future when this arrangement is used to measure currents. Each column is the 4 readings from a particular meter for the 4 currents. Each row is the readings from the 4 meters for a particular current. The matrix A of readings is:
[[ 0.95 1.09 1.01 1.09 ] The column matrix of currents is:
[[ 1 ] What we want to do is get a vector W which will contain 4 numbers, weighting factors to be applied to the meter readings. In other words, if the 4 meter readings are m1, m2, m3 and m4, and the weighting factors (elements of W) are w1, w2, w3 and w4, then the derived current value will be w1*m1+w2*m2+w3*m3+w4*m4. Obviously, given that the 4 meters are nominally identical, weighting factors of w1=.25, w2=.25, w3=.25 and w4=.25 are close to what we want. Since the meters will have some unavoidable manufacturing tolerances, the weighting factors will be slightly perturbed from the ideal values I just gave. Other solutions to the problem would be W vectors of (the T means transpose): [ 1 0 0 0 ]T, just use the readings from the 1st meter. [ 0 1 0 0 ]T, just use the readings from the 2nd meter, etc. [ .5 .5 0 0 ]T, just use the readings of the 1st and 2nd meters. [ .5 0 .5 0 ]T, just use the readings of the 1st and 3rd meters, etc. If we solve the A*W = B linear system on the HP48G (or its descendants), we get:
[[ .8822598871 ] While this is an exact mathematical solution, it doesn't make sense given the physical situation. Two of the weights are negative! And the system isn't particularly illconditioned; the condition number of the A matrix is about 1000, so we lose about 3 digits of accuracy in the solution. What we would like is 4 weights of about .25 each. Here is a case where the exact mathematical solution doesn't make sense. It reminds me of those freshman physics problems where you get two solutions to a problem, one of which doesn't fit the physical situation.
What shall we do?
Re: Solving a linear system with physical meaning  John Keith  01202008 Hi Roger,
Re: Solving a linear system with physical meaning  Rodger Rosenbaum  01202008 Just to clarify, the data aren't real measurements. I started out with an ideal matrix and added random perturbations. Any trend is accidental, an artifact. So any method of solution intended to be general shouldn't make special provisions for apparent trends in these particular data. This whole thing is intended to be illustrative of the HP48G user manual advice that sometimes the exact solution might not be what you want.
The question is, how SHOULD we handle the problem?
Re: Solving a linear system with physical meaning  Jonathan Eisch  01202008 It seems to me that if you have four meters, with random errors (uncorrelated with any known quantity), any combination of these meters (with weights equaling one, of course) are equally valid. Garbage in, garbage out. In other words: Why would you expect [.25 .25 .25 .25] when [1000 999 0 0] is just as good? It seems to me that you would have to minimize something to deal with your random errors. If your errors were linear with respect to current, then I would guess that your system makes some sense. Why not try that?
Jonathan
Re: Solving a linear system with physical meaning  Rodger Rosenbaum  01202008 Quote: The idea is that if the errors are zero mean, uncorrelated with each other, then if you include more meter readings, the desired signal (actual value of the current) is accumulated while the errors cancel each other, in a manner of speaking. That is why [.25 .25 .25 .25]T is better. "Minimizing something" is as easy as just adding up weighted readings from a bunch of meters, the more meters the better. My point here is to show that the standard way of solving a linear system doesn't give a good result in a case like this. Imagine if the A matrix were:
[[ 1 1 1 1 ] If you try to solve the system in the usual way, you get an error. But, plainly, the solution [.25 .25 .25 .25]T is the desired one. (There an infinite number of "exact" solutions in this case.)
And, in fact, solving this system with the LSQ function on the HP50 gives just that result. But, it won't work with the A matrix I gave in my first post in this thread.
Re: Solving a linear system with physical meaning  Dave Shaffer (Arizona)  01202008 This is NOT a linear system  as John pointed out, your meter readings are non linear (and, I suspect, if you made a plot also have nonzero intercepts, which I think you have not properly accounted for either). So, certainly the manual's warning is more than a propos about blinddly accepting a "solution"!
If you found least squares fits for the behavior of each meter, along with some error estimates, you could probably formulate a much better and more legitimate averaging routine.
Re: Solving a linear system with physical meaning  Jonathan Eisch  01202008 Quote: No, you got exactly what you asked for. (A great result!) You wanted some set of weights to make the readings you provided equal the values you provided. If you do the matrix multiplication with the weights that you got, you'll see that you get 1,2,3,4 to within the precision you provided. That looks like success to me. If you do the matrix multiplication with the .25,.25,.25,.25 matrix you want, you get something like 1.035, 1.955, 2.9575, 3.9675 with is clearly further away, so clearly not a mathematically better set of weights.
Quote: Mathematically you'd be minimizing a flat parameter space, nothing matters, so no solution would be mathematically preferable over any other. In other words, numbers don't /desire/ anything. ;) But you didn't use this perfect A matrix, you used a slightly different one, so now your flat space has tiny bumps, and your minimization rolled into one of them. Try giving it a (linear) slope and see what you get.
Jonathan
Re: Solving a linear system with physical meaning  Rodger Rosenbaum  01212008 Quote: As I mentioned in the first post, we're trying to solve w1*m1+w2*m2+w3*m3+w4*m4 = b1, etc. Like many a least squares problem, the problem is linear in the weights, although this is not a least squares problem since we have 4 equations and 4 unknowns.
The meter readings are not nonlinear; they are linear with the addition of noise. That is the sort of situation where averaging of some sort can reduce the effect of the noise.
Re: Solving a linear system with physical meaning  Rodger Rosenbaum  01212008 Quote: I failed to be sufficiently explicit in my first post in describing what this is all about. The idea here is to find some weights with the "calibration" measurments I described in the first. Then subsequent measurements would be used on another day to measure some currents, and the derived weights would be applied to those subsequent measurements. The current(s) to be measured would must likely be different than the initial "calibration" currents, and would not get the nearly exact (to about 10 digits) results you get if you apply the weights:
[[ .8822598871 ] To be more concrete, let's suppose I make some more measurements to see how well the weights we determined in the first set of measurements work with some new current measurements. The ideal currents applied this time will be:
[[ 1.5 ] and the measured currents might be (perturbed from the exact values with random noise):
[[1.44 1.42 1.41 1.54 ] The weights calculated with the standard linear system solver are:
[[ 5.8890631686 ] If we apply these weights to the second set of measurements just above, we get about 10 digit agreement with the applied currents. But, if we apply the weights derived in the first post, namely:
[[ .8822598871 ] we get (rounded to 6 digits):
[[ 1.12995 ] If we just use the intuitively reasonable weights [ .25 .25 .25 .25 ]T, we get:
[[ 1.4525 ] considerably better. And perhaps we can do even better. The idea is to find some weights such that if applied to current readings contaminated by noise (errors), the result won't be perturbed much away from the true value. Using the weights derived from an exact linear system solution is not a good thing; these weights will magnify the noise by an amount about equal to the condition number of the A matrix used to derive the weights. The reason it isn't good is because this is a system where the columns of the A matrix are almost carrying the same information (almost linearly dependent). This means that the condition number of the matrix is likely to be large. An exact solution of the system works well for the A matrix used in the solution, but another matrix with different perturbations (additive noise) may not give a reasonable solution. One way to deal with this (sometimes recommended for illconditioned systems) is to reduce the order of the system. Just delete the columns that are linearly dependent on the others. In this case, just select one of the columns. But, even though the columns are nearly linearly dependent, there may be some information to be gained if all of them are included. The problem is how to do this without getting the error magnification that using standard linear system solution methods gives.
The physical nature of this particular problem shows that weights of [.25 .25 .25 .25 ]T are probably close to the ones we want.
I said: Quote: Then you said:
Quote:
Look again at the subject line. It's not true that "nothing matters" given the physical nature of the setup. The reason for preferring the [ .25 .25 .25 .25 ]T weights is not mathematical, or because the numbers "desire" something. When I said "desired" solution, I was referring to my desire, not the desire of the numbers. The reason is the physical situation. A person could just select one of the meter readings and just ignore the rest, but then you would be throwing away possible information carried by the other readings. If you use the [ .25 .25 .25 .25 ]T weights, you are averaging the readings and presumably reducing the effect of the noise. Because I chose to use identical meters in this hypothetical situation, it's easy to see approximately what the weights should be. Similar situations may arise where it isn't so obvious. What mathematical procedure can we follow to get a good set of weights, less affected by the illconditioning of the A matrix?
Re: Solving a linear system with physical meaning  Jonathan Eisch  01212008 Quote: Averaging?
Jonathan
Re: Solving a linear system with physical meaning  Nick  01212008 Rodger, this doesn't have to do with mathematically exact solutions that aren't possible physically, like for example partcles with imaginary mass fulfilling special relativity and travelling at speeds greater than c. The result that you got is: 1) Absolutely valid for itself, since there is absolutely no reason for weights to be always positive. Even absolute temperature is "formally" accepted to be "negative" (i.e. under 0K) when used only as a "weight" for entropy differences. 2) Even if we want all weights to be positive, your underlying mathematical model doesn't include this as an additional constraint, and hierin lies the problem, as Jonathan also described very nicely with "bumps". You know that all weights have to be the same, but your model doesn't and so it goes down the bumps. You must explicitely formulate the constraint that w1=w2=w3=w4=1/n where n is the number of your current meters, or anything else in a repeated measurement. You do statistics with a constraint here, but you don't formulate the constraint explicitely  or using other words you want a flat world but your mathematical model has bumps and no breaks to prevent the "moving things" (weights) to fall into the bumps as they search for their "best" values. (The word "best" means arithetically best here. So it is not a subject of mathematical solutions being physically impossible  it is a matter of inadequate mathematical description of the system iff we demand all weights to be positive, and even more so iff we demand them to be all positive and all equal to each other. I try to present an idea here using as few words as possible and not having any possibility (or knowledge ;)) for using textbook notation for formulae. Let's reduce our sight to one of the current meterings, say A1 (1 Ampere). In the ideal case of ideal meters all meterings would be A1=A=1, the "real" current. So the first row of A would be [1 1 1 1] in its ideal version, and the corresponding ideal element of B would be also 1. Then, for the first row of the matrix equation we have: SUM(A1i*Wi,i=1..n)=B1 in the most general case, and: SUM(A1*Wi,i=1..n)=B1 in the special "ideal" case. Here we are in the flat world that Jonathan mentioned. Infinite many solutions as you also mentioned and even if you know about some of them, that "feels" better, we didn't told our system about our feelings yet, since the constraint is missing! Iff for all Wi it is valid: Wi=W=1/n, then the equation for the first row goes to: SUM(A1*W,i=1..n)=B1 => SUM(A1,i=1..n)*W=B1 => n*A1*W=B1 which for n=4, A1=1, B1=1 has one solution W=1/4. Notice that we get that solution, only *after* introduction of the contstraint! Let's move on to the nonideal case, where A11, A12, A13, A14 are different. We can formulate each of them as the "real" A1 plus some Delta that can be positive or negative. Denote that with DA11, DA21, and so on. Then for the first row again: SUM(A1i*Wi,i=1..n)=B1 Which means that: SUM((A1+DA1i)*Wi,i=1..n) = B1 => SUM(A1*Wi,i=1..n)+SUM(DA1i*Wi,i=1..n)=B1 This world has bumps and the Wi's will fall in there if we not explicitely specify again for all Wi: Wi=W=1/n . After this constraint we have: SUM(A1*W,i=1..n)+SUM(DA1i*W,i=1..n)=B1 => SUM(A1,i=1..n)*W+SUM(DA1i,i=1..n)*W=B1 => n*A1*W+SUM(DA1i,i=1..n)*W=B1 => (n*A1+SUM(DA1i,i=1..n))*W=B1 which again will have a solution of somewhere around W=1/n since the sum of all Deltas, SUM(DA1i,i=1..n) , is about zero. Again we get this result *after* introduction of the constraint! Putting all As and Ws and Bs into a matrix equation is much the same, but still the constraint *has* to be specified explicitely, in order to have an adequate mathematical description of what we want. Actually the matrix equation goes over to vector equations then, where the constant W can be calculated to 1/n (or 1/4 in your specific case). Nonetheless, as already said, there is also absolutely nothing wrong with negative weights, should the nonrestricted model be considered good enough. Thanks a lot for this very interesting problem  a very good demonstration of how some "intuitive feeling" can slip into mathematics and cause surprises if the formalism is not describing exactly what it was intended to desccribe. As Russel once said: The greatest challenge to any thinker is stating the problem in a way that will allow a solution. ;) But then again, all we intuitively considered true proved to be false! (From Russel too?) ;) Cheers and excuse my lust for maths late in the night!
Nick
Re: Solving a linear system with physical meaning  Karl Schneider  01222008 Hi, Rodger  Interesting problem  one which I'd never considered. The determinant of "A" (the matrix of randomerrorperturbed measurements) is only 0.0011, making it nearsingular. However, a matrix of four identical measurements from each of the four meters would be singular. Clearly, the linear solution of this exactlydetermined system is the wrong approach for specifying the weights. This linearalgebra solution, of course, is the one that enables each of the four data points to be matched exactly, but as you showed in a subsequent example, it produced poor results for a different set of quasirandom data. Since the measurement error is random, calibration of the meters should not be expected to provide any benefit, and it's likely impossible to mathematically derive an "optimal" solution for all expected measurements, based upon only the four provided. Each of the meters has comparable accuracy, so setting each weight to 0.25 is the reasonable thing to do. As a mathematical exercise, you might try generating additional "measurements" perturbed by an equal degree of random noise for each meter, and see how well the HP48G's leastsquares solution for overdetermined systems works. I'd bet that as more measurements are represented in the system, all the weights approach 0.25. Regards, KS
(Edited to sharpen one statement.)
Edited: 22 Jan 2008, 11:45 p.m. after one or more responses were posted
Re: Solving a linear system with physical meaning  Rodger Rosenbaum  01222008 Quote: It may not have to do with "particles with imaginary mass", but it most certainly has to do with mathematically exact solutions being unsatisfactory for reasons having to do with the physical aspects of the problem. That is what I intended when I started this thread, and I hinted at that in the subject line. Let me give a concrete example that I also hinted at in the first post. Consider this freshman physics problem. From a point on a platform, 10 feet above the ground, launch a steel ball upward with an initial velocity of 60 feet per second (perhaps at a slight angle so it doesn't fall back on your head!). How many seconds does it take until the ball reaches the ground? Taking the acceleration of gravity as 32.17 f/s, we set up the following equation: 16.085*t^2  60*t  10 = 0 Solving, we get two solutions, t=3.89 and t=.1598 Do we really believe that the ball will hit the ground .1598 seconds after we launch it, and again in 3.89 seconds? The problem I have given in this thread is similar in that there are numerous solutions that seem reasonable along with the one mathematically exact solution. We must look to the physical nature of the problem to help us select the appropriate solution. Let me digress.
In the November 1970 issue of The American Mathematical Monthly, there is an article titled "Pitfalls in Computation, or Why a Math Book isn't Enough", by Professor G. E. Forsythe. He gives an example: Here is a small linear system, A*x = B:
[[ .780 .563 ] * [[ x1 ] = [[ .217 ] Two solutions to the system are proposed, [ .999 1.001 ]T and [ .341 .087 ]T. Which one is better? The usual check is to substitute them both into the original problem. Substituting [ .999 1.001 ]T gives a residual ( A*xB ) of [ .001343 .001572 ]T. Substituting [ .341 .087 ]T gives a residual of [ .000001 0.0 ]T. It seems clear that the second proposed solution is better than the first, since it makes the residual far smaller. However, in fact the true solution is [ 1 1 ]T as the reader can easily verify. Hence, the first proposed solution is far closer to the true solution than the second.
A persistent person may ask again: which solution is really better? Clearly, the answer must depend on one's criterion of goodness: a small residual, closeness to the (mathematically) true solution, or something else. Surely one will want different criteria for different problems.  The problem I gave has physical aspects that lead to certain choices for a solution. First of all, the meters are nearly identical. That leads to the notion that they should be weighted nearly equally. Also, I assumed that the noise (errors) is zero mean, uncorrelated between meters. This is a reasonable assumption, absent any evidence to the contrary. In a real situation, there might be reason to assome otherwise. If so, then it will have to be dealt with some other way, but for the purposes of this thread, that's the assumption. Because the noise tends to be reduced by averaging processes, whereas the desired quantity (value of the current) isn't, we don't want to do what Jonathan Eisch suggested:
Quote: The reason the weights [1000 999 0 0] are not just as good is because those weights will greatly magnify the errors with respect to the desired current reading. Plus, of course, the readings from two of the meters are thrown away, losing any helpful information they may carry. The weights are mathematically equivalent if the current readings are identical for each meter, but that's not the case here. And besides the purely physical consideration that we would like to include the readings from all 4 meters (otherwise why bother having 4 meters?), there is a mathematical reason: In this problem, the sum of the weights in any of the solutions we've seen so far is nearly 1. It is well known that, given 4 numbers whose sum is 1, the rootmeansquare value of the 4 is a minimum when they are all 4 equal. Since the additive noise in the readings is uncorrelated, a weighted sum of the 4 noise signals will be minimized when the weights are nearly equal. So, let's assume that we have determined that the weights should be nearly [ .25 .25 .25 .25 ]T, which are far from the weights the exact linear system solution gives. It would appear that our criterion should be to judge the goodness of our solution by looking at the norm of the residuals. Calculate this on the HP50 by using the ABS function applied to (A*wB). Let's see if we can find the minimum residual norm solution where the weights are constrained to be equal. First so we will have a basis for comparison, let's compute the residual norm with weights of [ .25 .25 .25 .25 ]T (using the A matrix I gave in the first post of this thread). I get .07818. To calculate the residual norm where the weights are constrained to be equal, multply A by [ 1 1 1 1 ]T and get [ 4.14 7.82 11.83 15.87 ]T. Put this on level 1 of the stack and put [ 1 2 3 4 ]T on level 2 of the stack; press LSQ and get [[ .252607010711 ]]. Using [.252607 .252607 .252607 .252607 }T as a solution vector, compute the residual norm. I get .0540137, a substantial improvement over .07818. It is possible to do even better if the individual weights are not constrained to be equal. So far, nobody has given much in the way of numerical solutions. There have been discussions of various procedures, but no examples to show if they give a "good" result, with "good" meaning the sort of thing I've discussed. Nick said:
Quote: There are several things wrong with negative weights, and I hope I've explained why. Now, please use your calculators to find some weights with even lower residual norm, but close to [.25 .25 .25 .25 ]T, and tell us how you did it. But, be aware that there is a trade off. If you do a search for a global minimum in the residual norm, without constraints the search takes you to the exact linear system solution, with a residual norm of zero. It would seem that if you allow the individual weights to vary, you should be able to get a smaller residual norm, with the norm getting smaller as you approach the exact linear system solution.
What I'm looking for from all you who are interested is a procedure to find a set of weights with better residual norm than the one where the weights are constrained to be equal. And, try to do it with linear algebra techniques.
Re: Solving a linear system with physical meaning  Walter B  01222008 Hi,Rodger, let me second Karl and add my own experimental 20 MilliEuros : A single measurement means *nothing* unless you *know* the precision of your measuring instrument(s). In your starting case, you did assume a random measuring error for each instrument, which is perfectly correct. What can be calibrated (physically or mathematically as you want), however, is a systematic (= nonrandom) deviation between your instruments. This deviation you will only find by repeated measurements, since by doing this you can reduce the influence of the random factors. So, approaching the limit of an infinite number of measurements, your random errors will cancel and only the systematic deviations will survive. While you certainly don't want to measure infinite times, you may get reasonable estimates of the systematic deviations already after 5 measurements per instrument. After all, the basic law of statistics rules: With no information in you'll get no information out ;)
Best regards, Walter
Re: Solving a linear system with physical meaning  Valentin Albillo  01222008 Hi, Rodger: First of all, Happy New Year 2008 to everyone. And now ... Roger posted:
There's nothing in your mathematical model that accounts for the empirical fact that you know
If you want your mathematical model to agree with your intuition you need to include in
This will result in an overdetermined linear system of 10 equations in 4 unknowns, which
To that effect, I've
10 DESTROY ALL @ OPTION BASE 1 @ INPUT "#E,U=";M,N
which asks for the number of equations (E) and the number of unknowns (U), then lets If applied to your initial, nonoverdetermined system, it promptly gives:
>RUN
which are the exact weights you found so unexpectedly (give or take a few ulps). Now,
>RUNwhich are the kind of weights you were expecting to begin with. The moral:
lest you'll be surprised when thy results don't agree with thy expectations. Best regards from V.
Re: Solving a linear system with physical meaning  Rodger Rosenbaum  01222008 Hi, Karl. I was wondering if I would hear from you on this one! I'm also waiting to hear from Palmer.
Quote:
Sure enough. You can see that same thing happening if you just perturb the 4x4 A matrix enough times. I got my example matrix by starting with an ideal matrix, adding perturbations of 1%, and computing the condition number. I stopped when I got one with a medium high condition number. Most of the time the condition number's not that high, and the weights computed with a standard linear system solution are in the neighborhood of .25. I wanted a medium high condition number to show what would happen if the meters were nearly identical.
Re: Solving a linear system with physical meaning  Rodger Rosenbaum  01222008 Your comments are certainly something that a person would want to take into account, but they go beyond my point. I was just trying to point out that the exact mathematical solution to the simple situation I posed is not the "best" solution, and is less good the more the meters are identical.
Re: Solving a linear system with physical meaning  Antoine M. Couëtte  01222008 Wonderful, Valentin !!! You're back !! :) + :)) HAPPY NEW YEAR TO YOU TOO Antoine
Antoine M. Couëtte Edited: 22 Jan 2008, 8:12 a.m.
Why, thank you very much, Antoine ! :) [NT]  Valentin Albillo  01222008 Best regards from V.
Re: Why, thank you very much, Antoine ! :) [NT]  Antoine M. Couëtte  01222008 ... simply because I had not read from you on this forum for quite some time ( a few moons already ) ...
Re: Solving a linear system with physical meaning  Antonio Maschio (Italy)  01222008 Well, if you don't mind, could you explain to me what's the purpose of the SYS() instruction?
Taken from your program: Quote:10 DESTROY ALL @ OPTION BASE 1 @ INPUT "#E,U=";M,N  Antonio
Thanks in advance
Re: Solving a linear system with physical meaning  Valentin Albillo  01222008 Hi, Antonio: Antonio posteed: "Well, if you don't mind, could you explain to me what's the purpose of the SYS() instruction?"
N simultaneous linear equations in N unknowns.
For instance, MAT X=SYS(A,B) would solve the M systems with common coefficient matrix A and independent terms B, and
In the simplest case of solving a single system, M=1 and
Thanks for your interest and
Best regards from V
Re: Solving a linear system with physical meaning  Stefan Vorkoetter  01222008 Thanks Valentin for clearing this up and finding the real problem. I think what caused the belief that solving just the 4x4 system should be adequate was the fact that it was a 4x4 system. The fact that the system was square was arbitrary. After all, the number of calibration readings has nothing to do with the number of meters. It just so happened that 4 calibration readings were chosen for use with 4 meters, which resulted in a square matrix, which resulted in the belief that solving that 4x4 linear system should yield a meaningful answer. In other words, because the choice of the number of readings made the system look like a nail (even though it wasn't), it created a strong desire to solve it using a hammer.
Stefan
Re: Solving a linear system with physical meaning  Stefan Vorkoetter  01222008 Actually, is it necessary to add all six additional equations. Isn't "nominally identical" transitive, so it would only be necessary to add three of the equations? For example, [ 1, 1, 0, 0 ], [ 0, 1, 1, 0 ], and [ 0, 0, 1, 1 ]?
Stefan
Re: Solving a linear system with physical meaning  Walter B  01222008 Please consult the recent messages of Valentin and Stefan. Their posts will help you to find the point.
Re: Solving a linear system with physical meaning  Valentin Albillo  01222008 Hi, Stefan: Stefan posted: "Actually, is it necessary to add all six additional equations. Isn't "nominally identical" transitive, so it would only be necessary to add three of the equations? For example, [ 1, 1, 0, 0 ], [ 0, 1, 1, 0 ], and [ 0, 0, 1, 1 ]"
Using your 3 equations instead, the weights are also close to 1/4, but distinct from the ones using six equations and obviously "less balanced", i.e., more unequal. Check weight #1 (.2557) and weight #4 (.2492) for instance, versus .2532 and .2518 respectively.
Thanks for your appreciation and
Best regards from V.
Re: Solving a linear system with physical meaning  Valentin Albillo  01222008 Hi, Walter B.: Walter B posted: "Please consult the recent messages of Valentin and Stefan."
A puny joke, of course ! :) Thanks for your interest and
Re: Solving a linear system with physical meaning  Stefan Vorkoetter  01222008 Just an added observation: I just noticed that Valentin's computed weights don't add up to 1.0, but then realized that that is perfectly acceptable. Looking at the original input data, one column at a time, each meter individually tends to read low on average, so it's not surprising that the weight for each meter would be slightly more than 1/4.
Stefan
Re: Solving a linear system with physical meaning  Stefan Vorkoetter  01222008 Quote: The point is that you've computed the exact mathematical solution to the wrong problem, as Valentin points out below.
Stefan
Re: Solving a linear system with physical meaning  Karl Schneider  01232008 Hi, Stefan  You beat me to it! I saw that this morning, but didn't have time to prepare a post. Indeed, that is the reason for the elevated weights. The sum of the ten underestimates is 0.56; the sum of the five overestimates is 0.22, and one measurement was exactly correct.  KS
Edited: 23 Jan 2008, 12:13 a.m.
Re: Solving a linear system with physical meaning  Rodger Rosenbaum  01232008 Quote: I think you've attributed meaning to the "nominally identical" phrase that it doesn't have. Let me change my description of the meters. Suppose that the 4 (nearly perfect) meters have full scale readings of 5, 6, 7 and 8 amperes respectively. Now they are no longer indentical in any sense except that they read the same for any current except for a small random error. Also suppose that their readings are unaffected by any scale factor error, or any other systematic error. Assume the only errors are additive noise. I created the A matrix in the original post by starting with an ideal matrix with identical columns of [ 1 2 3 4 }T (which could have come from 4 perfect, identical meters, or just as well could have come from 4 perfect, nonidentical meters; I probably shouldn't have said "nominally identical". It seems to have engendered a complication the problem doesn't have) and adding random noise. It was the starting matrix of identical columns that I intended to imply by the phrase "nominally identical meters". Now what you called a "known fact" no longer obtains. How does this change how you will determine the weights? I don't agree that any set of weights is "mathematically and logically as acceptable as any other." Since the contaminating noise rendered the A matrix nonsingular there is only one such solution. And, the "nominally identical" phrase didn't make it untrue. Surely you wouldn't say there was no mathematically acceptable set of weights when the meters were "nominally identical", but now that the meters are 5, 6, 7 and 8 amperes full scale, there is? Or do you say that there is still no mathematically acceptable solution even with the different meters? I've reprinted Valentin's description with a literal variable replacing all the 1s in the last 6 rows:
Quote:A(1,1)? .95, 1.09, 1.01, 1.09 Now let's evaluate this overdetermined system with various values for k:
k Solution Note that if we take the original A matrix:
[[ 0.95 1.09 1.01 1.09 ] and post multiply it by:
[[ 1 ] we get:
[[ 4.14 ] Using this as an A matrix, and with our original B matrix, [ 1 2 3 4 ]T, we can solve as an underdetermined system. We get [[ .252607010711 ]]T. If we multiply [[ 4.14 7.82 11.83 15.87 ]]T * [[ 252607010711 ] we get [ 1.04579302434 1.97538682376 2.98834093671 4.00887325998 ]T. This is the solution where all the weights are constrained to be equal (I gave this solution in another one of my posts), as we shall see. If we divide [ 4.14 7.82 11.33 15.87 ]T by 4, we get [ 1.035 1.955 2.9575 3.9675 ]T. Now, create a new A matrix with 4 identical columns equal to the one we just calculated:
[[ 1.035 1.035 1.035 1.035 ] If we try to solve a system with this A matrix using a standard linear solver, we'll get an error because this matrix is singular, and there are an infinite number of solutions. However if we create a solution vector (W column matrix) with the value we calculated a while back, namely [ .252607010711 .252607010711 .252607010711 .252607010711 ]T, and use it to postmultiply this A matrix, we get [ 1.04579302434 1.97538682376 2.98834093671 4.00887325998 ]T, the same result as above. This is the same result Valentin's procedure gets when we let the value of k become large. Realizing that whenever we solve a linear system A*W=B, the data in the A matrix are constraints on the solution, we see that the additional constraints in Valentin's system when k = 1 are rather weak by comparing the value 1 to the other values in the first 4 rows. His constraints do tend to push the solution toward a state of equality among the columns, but they don't push very hard. The parameter k must be quite large to get substantial equality. I must say that I rather like Valentin's procedure. It provides a way to move continuously from the mathematically exact solution (with k=0) to the identical weights solution (with k=infinity as the limit), giving inbetween solutions by selecting various values of k. There is at least one other method, by the way. And there is no need to postulate "nominally identical" meters for the procedure to have merit. The procedure does its good work regardless of the nature of the meters. We need only know that the meter readings are from meters perfect in every way except for small random errors. There may be desireable properties for some of those inbetween solutions, such as less error magnification in exchange for greater residual norm. This is the very sort of thing I was trying to motivate people to explore, and it still hasn't been done.
Valentin gets a lot of credit for having actually done some calculations; now how about the rest of you guys? 
Valentin said: Quote:
Quote: I don't know why so many people keep attributing this to me. I never said or implied any such thing. I expected the solution (using a standard linear solver) to be just what I got:
[[ .8822598871 ] What I'm trying to get across in this thread is that most people would probably solve the system that way and think it was a good solution. I'm not one of those people. I've learned better and I'm trying to pass along what I've learned. I explained why this isn't a good solution for this particular physical configuration. It has some undesireable properties, and there are other approximate solutions with desireable properties, and the error due to the fact that they are approximate is of the same order as the input data. I then asked how to find them. So far, only Valentin has given a useful answer with numerical results showing how his procedure works.
Quote:
and Valentin pointed out that the result isn't quite the same. I would point out that if instead of using 1 in those 3 rows, use 10000 as I did in Valentin's system, and it becomes clear that as this parameter becomes very large, the solution tends to the same solution as Valentin's system when k is very large, which is the constrained equal weights solution.
Re: Solving a linear system with physical meaning  Rodger Rosenbaum  01232008 Quote: If you reread my posts, you will see that from the beginning I said that the mathematically exact solution was not adequate, and I asked for suggestions as to what might be a better solution, and how to find it. I was needling you guys to get out your calculators and play with some linear algebra.
Quote: It didn't just happen; I did it that way deliberately because that is just the situation which would lead people to the common belief that a standard linear system solution would be adequate.
Quote:
It may have resulted in such a belief for some people. I have learned to reject that belief and my whole purpose in this thread is to show why and when it should be rejected.
Re: Solving a linear system with physical meaning  Rodger Rosenbaum  01232008 This isn't unexpected, and it isn't that the meters are reading low. I only used 16 random samples when I created the perturbed A matrix, and I wouldn't expect them to have a zero mean. Not with only 16. :(
That's just like the real world; it takes a lot of samples for the mean to drift down to zero.
Re: Solving a linear system with physical meaning  Stefan K.  01232008 Quote:
Just to add my 2 cents (and summarizing what others in this thread alrady have pointed out):
You try to make nonexact measurements (due to nonlinearity of meter, reading and random errors...) exact by "calibrating" a set of 4 meters at 4 points (4 different currents)  that's impossible.
The way I would do it is linear regression: you try to find a linear function (y=k*x+d) with unknown coefficients k and d, and fit them by minimizing the error (sum(currenty)^2). Actually you end up with a set of linear equation agains (for k and d), which you can solve by inverting a matrix (but since it's 2x2, you can almost do it in your head).
Stefan
Re: Solving a linear system with physical meaning  Rodger Rosenbaum  01232008 Stefan, you're putting words in my mouth. Look at the beginning of my first post. I said: "Imagine that you need to measure an electric current, and to increase accuracy you will make the measurement with 4 nominally identical meters and derive some kind of weighted average of the readings of the 4 meters." I did not say, and I'm quoting you:
Quote: I never claimed to make nonexact measurements become exact. You said:
Quote: Exactly what are you referring to when you say "...the way you did it...". Do you mean the weights derived from a standard linear system solution? I only showed those weights as a bad set of weights. I specifically did not advocate their use. If you think you're rebutting my advocacy of those weights, your rebuttal is misplaced, because I don't advocate them. And again, from my very first post: "While this is an exact mathematical solution, it doesn't make sense given the physical situation. Two of the weights are negative!" Perhaps you should reread my first post.
Quote:
Why don't you do what Valentin did and what I did: work out an example and show us some numbers. Show that your technique would have a smaller residual norm than Valentin's weights, for example.
Let's see ...  Valentin Albillo  01232008 Hi again, Rodger:
Rodger posted:
"Let me change my description of the meters."
"Suppose that the 4 (nearly perfect) meters have full scale readings of 5, 6, 7 and 8 amperes respectively."
These meters are just simple analog panel meters.". I guess your new ranges for them mean that they are "nominally identical" no more as well ?. "I probably shouldn't have said "nominally identical". It seems to have engendered a complication the problem doesn't have)"
there aren't any. You started with an inadequate, faulty model and got counterintuitive, faulty results. Correct your mathematical model and all complications and weird results disappear in thin air. That's all there's to it. "It was the starting matrix of identical columns that I intended to imply by the phrase "nominally identical meters".
you said and now pretend that everyone copes instead with what you intended to say, "Now what you called a "known fact" no longer obtains. How does this change how you will determine the weights ?"
rational thing, which is to assume your "nominally identical" condition essentially means "equal weight" in terms of weights, which is what we're computing here.
It doesn't matter whether the meters are the same model and color or have different scales, ranges, or
That's what the improved model I suggested does. It explicitly specifies that every model should ideally
the one, which if they are good enough, will simply get you similar results and the choice between one model or the other is left to either additional considerations (which can be reflected in the model in order to make it even more accurate) or else personal criteria.
For instance, in my suggested model I translated your "nominally identical" condition to "every meter
But you could have also translated the constraint as "each meter is nominally identical to every w1 = 1/4, w2 = 1/4, w3 = 1/4, w4= 1/4and the leastsquares solution would reflect this fact and also compute reasonable weights that would tend to fit the readings and tend to be 1/4 as closely as possible.
The selection of the correct model is thus not necessarily unique, and different correct models
"This is the same result Valentin's procedure gets when we let the value of k become large [...]
actually nothing as it's only a case of what I've already stated: there are diverse correct models, and in absence of more information, the choice is almost personal, all of them will give very similar, good results.
In this case, by inserting a constant k in the extra equations implementing the "equal weights" presuposed
If you select k=0, you're telling the model: "I don't care for equal weights at all, please fit the
if you select k=1000 or greater, you're telling the model: "I care very little for the readings, please
"There may be desireable properties for some of those inbetween solutions, such as less error magnification in exchange for
you care for equal weights, I simply selected the more or less balanced value of k = 1, which gives some importance to both data fitting and nominal weight equality.
Absolutely nothing in your initial conditions gives the slightest clue about your personal position in
So, not being you, I can't tell which is which, and arbitrarily select k=1 as the most
You see, in the end it all depends now on your internal knowledge of the situation (how trustworthy
a Spanish idiom for this but it doesn't translate well. Best regards from V. Edited: 23 Jan 2008, 8:05 a.m.
Re: Solving a linear system with physical meaning  Stefan K.  01232008 Hi,
But since we already agreed that this is no good solution, here is how I would do it:
First define what B really is: it's some control parameter, it could be what you think the real current is, or just your voltage source value divided by 1 Ohm... Then you need a model, for simplicity let's assume that your current reading is linearly dependent on the control parameter,
[[B_i^2 B_i] [[k] [[A_i*B_i](you have to do the sums over the parameters with the index, N=4 is number of measurements). Once you have the coefficients for the polynomial, you can also calculate the error \chi^2 for every instrument, and use this as measure for your weighting factors. I suggest to use 1/(1+\chi^2) (I added 1 to the denominator in case the error becomes accidently 0, it could be also some other number), of course you have to normalize them so that they add up to 1. But since \chi^2 is always positive, you won't get unreasonable weighting factors. And this also gives you a sound way to get interpolated values.
A slightlybelated "welcome back" (matrix solutions)  Karl Schneider  01242008 Hi, Valentin  A sincere "Welcome back!" from your 12week hiatus, and thank you for the straightforward and clear analysis of the mathematical problem posed by Rodger. No obvious way to mathematically derive reasonable weightings was immediately apparent to me, but yours made perfect sense. This kind of insight is especially valuable.
Quote: Your elegant HP71 program also reminded me of a theorem of linear algebra that needed to be "dusted off" after the passage of 12 or 17 years since I'd seen it. Perhaps other readers would benefit from an interpretation and explanation:
The program utilizes the theorem that, for an overdetermined and inconsistent linear system Ax = b, On pages 1414, 1415, and 1812 of the HP48G Series User's Guide, we see that the HP48G will automatically return optimized solutions for systems that are not exactlydetermined:
So, with the "LSQ" (least squares) function, the HP48/49/50 will automatically perform such optimizations, which must be done manually on an HP15C, HP42S, or (as Valentin did) on an HP71B.  KS
Edited: 27 Jan 2008, 5:11 p.m. after one or more responses were posted
Solving a linear system: Calculations  Karl Schneider  01242008 Quote: Well, for what it's worth, I calculated the determinant and system solution of your original problem on an HP42S, and got similar results within a few ULP's of those from your HP48G* and from Valentin's HP71B/Math ROM, but matching neither. It seems as though HP was continually refining the linearalgebra code for the Saturnbased machines. As far as I'm concerned, Valentin's solution handles the problem you posed to the furthest justifiable extent. If I had available simultaneous and nominallyidentical measurements of a single quantity, I'd calibrate the devices and weight the measurements equally. If I wanted robustness from redundancy, I'd probably buddycheck the measurements against each other, throw out any bad one(s), and weight the good ones equally. Regards,  KS
Edited: 24 Jan 2008, 3:11 a.m.
Re: A slightlybelated "welcome back" (matrix solutions)  Valentin Albillo  01242008 Hi, Karl:
Karl posted:
Thank you very much for both your fond welcomeback and your kind comments re my analysis, much appreciated. As for the "12week hiatus", I've been dedicating my free time to some other hobbies o'mine having little to do with calcs or math and much to do with fine arts and ancient cultures (nihon), all in all extremely rewarding (if expensive).
I did occasionally read this forum but was unlucky to find it infested by the usual suspects and their moronic ramblings and tantrums, and seriously lacking in interesting contents so I didn't stay for long. It saddens me to say this but frankly, IMHO, the magic is decaying at an accelerated rate and I foresee my participation diminishing accordingly. :( "Your elegant HP71 program [...]"
10 DESTROY ALL @ OPTION BASE 1 @ INPUT "#E,U=";M,Nwhich is shorter, slightly faster, and takes much less RAM, thus it's more "elegant".
Thanks for your interpretation and explanation, which I'm sure most readers will welcome, and
Re: Let's see ...  Rodger Rosenbaum  01262008 A computer crash kept me from responding for a couple of days.
Quote: Are you suggesting that all of the various sets of weights obtained as the parameter k varies from 0 to inf all give similar results? What would be the "results" which you would expect of the weights, to what use would you put them, that leads you to say that "all of them will give very similar, good results." What mathematical test would you use to show that "all of them will give very similar, good results."?
Re: Solving a linear system: Calculations  Rodger Rosenbaum  01262008 Quote: The problem I gave was simple and symmetrical on purpose, and if the goal was just to solve that particular problem, no powerful mathematics is necessary at all. I gave the solution [ .25 .25 .25 .25 ]T in my first post. And as you say:
Quote: If the errors are small enough that the nature of the data {nonimally identical measurements) is apparent by visual inspection, the appropriate weighting can probably be determined without doing any matrix arithmetic at all. But surely there must be some criterion for the "goodness" of a solution that is a little more mathematical than just how it looks (how close is a solution to the [ .25 .25 .25 .25 ]T solution). Do you have any ideas? But what if the system is not so symmetrical? Or what if it has a singular A matrix? Can we find an approximate solution? I'll give a system here and start a new thread about it this weekend. Given A:
[[ 4. 2. 3. 3. ] and B:
[[ 22.1743265837 ] Solve the system A*X=B
Re: Solving a linear system: Calculations  Stefan Vorkoetter  01262008 Quote: My Matrix Multitool program computes a determinant of 6.63e10 and an X vector of [8.1365, 17.4676, 26.5950, 20.0000]. However, the actual determinant is 0, so the system is singular and cannot be solved. Stefan
Re: Solving a linear system with physical meaning  PeterP  01262008 simply beautiful, Valentin. Cheers
Peter
Re: A slightlybelated "welcome back" (matrix solutions)  PeterP  01262008 Thanks Karl, I always learn lot from your posts! Cheers
P
Re: Solving a linear system: Calculations  PeterP  01262008 for what its worth Roger, thanks for posting an interesting problem, this was a good thread again (albeit it seems we still like to move to bickering in between :) Thanks again! Cheers
P
Re: Solving a linear system: Calculations  Karl Schneider  01262008 Rodger 
Quote: How about this: If each meter or transducer gives a reading or output result equal to the following: The exact value + an uncorrectable calibration error + random noise of uniform distribution, Then, one should be able to calculate the respective probabilities of each device providing the most accurate result for any given measurement. The weights could be set equal to those probabilities. This is more of a statistical exercise than a linearalgebra problem, and I would say that a reference on metrology should be consulted.
Quote: The singular matrix A defines an underdetermined system, so there are infinitely many solutions. The HP48/49/50 can return the one with the lowest Frobenius (Euclidian) norm, as indicated on page 1415 of the HP48G Series User's Guide: On my HP49G, I got [2.01952510743 2.23362533599 1.20500296581 2.00465552820]^{T} as that solution for X. A*X produces each value of your constant matrix B to within one ULP. Note how much lower in "pythagorean magnitude" this solution is, compared to Stefan's. The HP48/49/50's capability of calculating exact determinants for integervalued matrices allows it to identify a singular (underdetermined) matrix, so that the optimization can be performed. The HP15C, HP42S  and Stefan's program  produce very small nonzero results for the determinant. In the case of the HP calculators specifically, roundoff errors from calculation of the LU decomposition are to blame.  KS
Edited: 26 Jan 2008, 6:04 p.m.
Re: Solving a linear system: Calculations  Rodger Rosenbaum  01262008 Just because the determinant is 0 doesn't meant the system has no solution. In this case there are an infinite number of solutions. If flag 22 is clear on the HP50, using the technique of putting B on level 2 of the stack, A on level 1 and pressing /, you get an error. On the other hand, if flag 22 is set, then the HP50 gives:
[[ 1628.13646975 ]
It's having the same problem as your program. Its internal LU decomposition is probably not getting a determinant of exactly zero.
Re: Solving a linear system: Calculations  Rodger Rosenbaum  01262008 Quote: This is a fine idea, but there is much more to be said for the simpler case of random noise only. I think it's well to know how do deal with that case effectively first, and then the further complications.
Quote: Even though HP calls this an underdetermined system, I think that could be confusing. Since there are 4 equations and 4 unknowns, it looks to be an exactly determined system at first glance. Because 1 of the columns is a linear combination of the other 3, the system is technically underdetermined, so it's not incorrect to call it that, but I think there is a better description. The terminology used in the mathematical literature for this kind of system is "rank deficient" if the A matrix is not square, and "singular" if it is. I sometimes use the term "rank deficient" even for a square matrix, because that's what it is, and this term reminds us of what we must do to deal with it.
Quote: On my HP50, I get exact equality. Do you get 1 ULP differences for some of the values? I assume you used the LSQ command to get this result. It is well to note that flag 54 must be clear to get the result you got. It's interesting to see what you get if you set that flag.
Quote: The HP48G and descendants correctly calculate determinants of integervalued matrices (where the elements are approximate numbers) by virtue of setting "tiny elements" to zero, according to the setting of flag 54. This typically doesn't need to be done for small integer matrices, and in fact for this one, the setting of flag 54 doesn't matter, even though the elements of the matrix are approximate, not exact, numbers. For exact numbers, an integer determinant will be returned for any matrix of only integer elements, regardless of the setting of flag 54, and I sometimes use this as a check. This doesn't apply to the LSQ command, anyway. LSQ's method of solution isn't described in either the HP48GX user's guide or the AUR. However, a third party book titled "Linear Algebra Investigations with the HP48GX, by Donald LaTorre, says that the LSQ function uses orthogonal decompositions, either the QR or the LQ decomposition. (LaTorre was involved in the design of the HP48G.) The determinant doesn't enter into it, but the setting of flag 54 does have an effect.
I'm going to start a new thread to continue this.
Re: A slightlybelated "welcome back" (matrix solutions)  Thor Lansen  01262008 Welcome back my friend Valentin! in the words of Mark Twain, I am glad to hear the reports of your death had been greatly exaggerated.
Regards, Thor
Teminology of matrices and linear systems  Karl Schneider  01272008 Rodger  Upon further review of my linearalgebra textbook as well as the primary HP48G reference, it seems that each of us was not always using strictlyorthodox terminology of matrices and linear systems in all cases. A few things ought to be cleared up. You stated,
Quote: The term "rank" applies only to a matrix. The rank of a matrix is the dimension of its range. The rank cannot exceed the number of rows or the number of columns of a matrix, whichever is fewer. A matrix can be not square, and still be fullrank. "Rankdeficient" is equally applicable to square or nonsquare matrices. Of course, only a square matrix can be singular. Overdetermined, underdetermined, and exactlydetermined apply only to systems, e.g., Ax = b. It also seems counterintuitive to me that an underdetermined system can be made "exactlydetermined" or even "overdetermined" by simply padding it with nonindependent rows, but... oh, well. An overdetermined linear system is defined in my text, as well as in pages 1414 and 1415 of the HP48G Series User's Guide, simply as one which has more equations than independent variables. However, it is possible for an overdetermined system to have no solution, one unique solution, or infinitely many solutions  it depends on the rank of the system matrix, and whether the system is consistent. So, to reiterate the basic conclusions:
I can't say with certainty what kind of solution(s) could be obtained with an overdetermined and inconsistent system having a rankdeficient matrix.
BTW, flag 54 was indeed clear (the default setting) on my HP49G when I obtained the leastsquares "LSQ" solution to your exactlydetermined system with the singular (thus, rankdeficient) matrix:
2.01952510743 This solution, when multiplied by your matrix A, produced your vector B with an error of one ULP on two elements, and was exact on the other two. I got a very different result from "LSQ" with flag 54 set:
673.501526294 This solution, when multiplied by your matrix A, produced a residual from B of 10^{10}*[263 368 275 493]^{T}.  KS
Edited: 28 Jan 2008, 3:55 a.m.
Re: Teminology of matrices and linear systems  Rodger Rosenbaum  01282008 Quote: You got me! I was in a big hurry when I wrote that post and I left out the word "matrix" after the word "system" in the second sentence. I sometimes see the A matrix called the "design matrix" also. Maybe I'll just call it a "square" matrix when there as many equations as variables. On rereading those two sentences, I realize I left out something. A rank deficient square system may have two natures depending on whether the rectangular matrix which is the A matrix augmented by the B matrix has a different rank than the A matrix alone. In one case it has an infinite number of solutions, and may be described as underdetermined. In the other case, it has no solutions (is inconsistent) and may be called overdetermined. So a system with a square A matrix may be over, exactly, or underdetermined in this sense.
Quote: Actually, the User's Guide says, "These systems have more linearly independent (emphasis mine) equations than independent variables." That "linearly independent" part is crucial. A system with more total equations than independent variables could have more or fewer "linearly independent" equations than independent variables.
Quote: I'm going to have to think about this one. Let's say you need to delete 3 rows to make the system square. Is the solution to the system with the deleted rows the same no matter which rows are deleted? ================================================================= I get slightly different numerical results sometimes! I'm going to have to try this out on an HP48G and also the HP49G. For example:
Quote:
I get 10^10*[223 368 235 753]T on the HP49G+.
Re: Teminology of matrices and linear systems  Karl Schneider  01292008 Quote: I'm still struggling to reconcile my textbook's complete statement with HP's statement. Both might be correct, if properly interpreted (which I might not have done). There are different ways of stating things...
Quote: Good point  no, the square "reduced" matrix would need to remain fullrank, leaving a basis for the row and column space intact. In other words, only rows whose removal would not reduce the rank of the matrix should be deleted. For example, to leave two rows in the matrix that are merely scalar multiples of each other would reduce its rank.  KS
Edited: 29 Jan 2008, 2:25 a.m.
