Perhaps the first significant observation in the theory of infinite series is that there are convergent series whose terms can be rearranged to form a new series that converges to a different value.
A well known example is provided by the alternating harmonic series,
and its rearrangement
Henry Parker Manning. Irrational numbers and their representation by sequences and series. John Wiley & Sons, 1906,
Laurent evaluated the latter by inserting parentheses (see pages 97, 98):
A similar argument is possible with the rearrangement
which can be rewritten as
The first person to realize that rearranging the terms of a series may change its sum was Dirichlet in 1827, while working on the convergence of Fourier series. (The date is mentioned by Riemann in his Habilitationsschrift, see also page 94 of Ivor Grattan-Guinness. The Development of the Foundations of Mathematical Analysis from Euler to Riemann. MIT, 1970.)
Ten years later, he published
G. Lejeune Dirichlet. Beweis des Satzes, dass jede unbegrenzte arithmetische Progression, deren erstes Glied und Differenz ganze Zahlen ohne gemeinschaftlichen Factor sind, unendlich viele Primzahlen enthält. Abhandlungen der Königlich Preussischen Akademie der Wissenschaften von 1837, 45-81,
where he shows that this behavior is exclusive of conditionally convergent series:
Theorem (Dirichlet). If a series converges absolutely, all its rearrangements converge to the same value.
Proof. Let be the original sequence and a rearrangement. Denote by and their partial sums, respectively. Fix . We have that for any , if is large enough, then for all there is some with . Also, there is a such that for all there is a with , so
Choosing large enough, and using that converges, we can ensure that the two displayed series add up to less than . This gives the result.