For a manifold $M$, the $k$'th order tangent bundle $T^kM$ of $M$ consists of all equivalence classes of curves in $M$ which agree up to their accelerations of order $k$. It is proved that at the presence of linear connection on $M$, $T^kM$ admits a vector bundle structure over $M$ and every Riemannian metric on $M$ can be lifted to a Riemannian metric on $T^kM$ \cite{Suri Osck}. In this paper, we construct the principal bundle of orthogonal frames $\O^kM$ of the Riemannian vector bundle $T^kM$ over $M$ and we prove that it is the associated bundle to $T^kM$ with respect to the identity representation of $\O(\E^k)$. Then, we develop a generalized principal bundle structure for $\O^\infty M$ associated with $T^\infty M=\lim T^kM$ by a radical change of the notion of the classical bundle of orthogonal frames and replacing $\O(\F)$ by a generalized Fr\'{e}chet Lie group.
