Algebra, Differential Calculus, Integral Calculus, Real Analysis, Differential Equations, etc are the main topics covered under IIT JAM Mathematics Exam. One can also expect questions from areas like Probability and Statistics, Complex numbers, etc too for this exam. There are plenty of books available which can be referred for studying in depth.

Books on Algebra

• Abstract Algebra by Bhattacharya, Jain & Nagpal
• Complex numbers from A to Z, by T.Andreescu,  D.Andrica
• Elementary Linear Algebra by Chatles W.Curtis
• Elementary Linear Algebra by H.Anton
• Fundamental structure in Modern Algebra by R.S. Mishra and N.N.Bhattacharya
• Linear Algebra and its Applications by David C. Lay

Books on Calculus

• Calculus by M. J. Strauss, G. L. Bradley and K. J. Smith
• Calculus by H. Anton, I. Bivens and S. Davis
• Integral Calculus by Gorakh Prasad
• A text –Book of Vector Calculus by Shanti Narayan
• Calculus by Pisknov

Books on Real Analysis

• A Course in Calculus and Real Analysis by Sudhir R. Ghorpade and Balmohan V. Limaye
• A course of Mathematical Analysis by Shanti Narayan
• A first course in Mathematical Analysis by Somasundaram, Choudhari
• Elementary analysis: the theory of calculus by K. A. Ross
• Fundamentals of Real Analysis by V.K Krishnan
• Introduction to Real Analysis by R. G. Bartle and D. R. Sherbert
• Mathematical Analysis by Apostol T.M
• Mathematical Analysis by Binmore K.G
• Methods of Real Analysis by Richard.R Goldberg
• Principles of Mathematical Analysis by Rudin .W
• Real Analysis by H.L. Royden

Books on Differential Equations

• Advanced Engineering Mathematics by Sastri S.S
• Advanced Engineering Mathematics by Wylie, C.R. and Burrett, L.C
• Differential and integral equations by Collins, P.J
• Differential equations and the calculus of variations by Yankosky
• Differential equations and their applications by Ahsan,Z
• Differential Equations by G.F.Simmons
• Partial differential equations – methods and applications by Mcowan, R.C

Books on Differential Calculus

• Differential calculus by Balachanda Rao and C.K Santha
• Differential Calculus by Gorakh Prasad

Books on Complex Numbers

• Complex variable by Murry R. Spiegel
• Complex variable – Theory and application by Kasana H.S
• Complex analysis by Ahlfors

Books on Probability and Statistics

• A Brief Course in Mathematical Statistics by Hogg, R.V. and Tanis, E.A
• A text Book of Statistics by C.E Weatherban
• An Introduction to Probability and Statistics by Rohatgi, V. K. and Saleh, A. K
• An Outline of Statistical Theory by Goon, A.M., Gupta, M.K. and Dasgupta, B
• Fundamentals of Mathematical Statistics by Gupta, S.C. and Kapoor, V.K
• Introduction to Probability Models by Ross, S. M
• Introduction to the Theory of Statistics by Mood, A.M., Graybill, F.A. and Boes, D
• Mathematical Statistics by Ray & Sharma
• Statistics by Murray & Spiegel

One can get these books in bookstores and some of the books are available as e-books which can be accessed via internet. They can also refer online study materials available for preparation.