The constrained Hamiltonian formalism is the basis for canonical quantization techniques. However, there are disagreements surrounding the notion of a gauge transformation in such a formalism. The standard definition of a gauge transformation in the constrained Hamiltonian formalism traces back to Dirac: a gauge transformation is a transformation generated by an arbitrary combination of first-class constraints. On the basis of this definition, Dirac argued that one should extend the form of the Hamiltonian in order to include all of the gauge freedom. However, Pitts (2014) argues that in some cases, a first-class constraint does not generate a gauge transformation, but rather "a bad physical change". Similarly, Pons (2005) argues that Dirac's analysis of gauge transformations is "incomplete" and does not provide an account of the symmetries between solutions. Both authors conclude that extending the Hamiltonian in the way suggested by Dirac is unmotivated. If correct, these arguments could have implications for other issues in the foundations of the constrained Hamiltonian formalism, including the Problem of Time. In this talk, I use a geometric formulation of the constrained Hamiltonian formalism to show that one can motivate the extension to the Hamiltonian independently from consideration of the gauge transformations, and I argue that this supports the standard definition of a gauge transformation without falling prey to the criticisms of Pitts (2014) and Pons (2005). Therefore, in order to maintain that first-class constraints do not generate gauge transformations, one must reject the claim that the constrained Hamiltonian formalism is fully described by the geometric picture; I suggest two avenues for doing so.