The notion of Gorenstein rings in the commutative ring theory is generalized to that of Noetherian algebras which are not necessarily commutative. We faithfully follow in the steps of the commutative case: Gorenstein algebras will be defined using the notion of Cousin complexes developed by R. Y. Sharp [Sh1]. One of the goals of the present paper is the characterization of Gorenstein algebras in terms of Bass numbers. The commutative theory of Bass numbers turns out to carry over with no extra changes. Certain algebras having locally finite global dimension are also characterized. The special case where the algebras are free modules over base rings is explored. Thanks to these observations, it is clarified how the Gorensteinness is inherited under flat base changes. In conclusion, a characterization for local algebras to be Gorenstein is given, accounting for the reason why the theory behaves so well in the commutative case. Examples are explored and open problems are given. See [GN2] and [GN3] for further developments.