Osmani was born in Dayamir, Sylhet District on 1 September, 1918.

Muhammad Ataul Gani Osmani, better known by General M.A.G. Osmani was the supreme commander of Mukti Bahini and Bangladesh Armed Forces during the Bangladesh Liberation War. He is currently revered as one of the greatest freedom fighters of the nation, a brave man never afraid of laying down his life.

He graduated from Sylhet Government High School in 1934. Later, he studied in Aligarh Muslim University, India.

In 1939, Osmani started his military career as a Gentleman Cadet of the Royal Armed Forces. Upon finishing his training at the British Indian Military Academy in Dehradun, he joined the Royal Army as a commissioned officer in 1940, when the World War 2 was going on. As a commander of a British Army battalion, he served in the Burma sector and was promoted to a Major in 1942. Following the war and the Partition of India in 1947, Osmani joined the Pakistan Army on 7 October, 1947, and was promoted to Lieutenant Colonel. He served as an aide to the Chief of General Staff in 1949. Later, he served as director of Rifle company in the 9th battalion of the 14th Punjab Regiment, additional commandant of the East Pakistan Rifles, and general staff officer of the army. Osmani was promoted to a Colonel in 1956 and held the post of deputy director of general staff and military operations. He retired from Pakistan Army on 16 February, 1967.

After his retirement, Osmani entered the politics of East Pakistan. He joined the Sheikh Mujib-led Awami League in 1970. As a candidate from Awami League, he was elected a member of the Pakistan national assembly in the 1970 elections in Pakistan. After the commencement of Bangladesh Liberation War, Osmani became a member of the provisional government.

With the formation of Mujib's government on 17 April 1971, Osmani was appointed commander in chief of all Bangladesh Armed Forces. Under his direct command, Osmani divided up the entire declared Bangladesh territory to 11 sectors.

After the country's independence, Osmani was elevated to the rank of General of the Bangladesh Armed Forces with effect from the date of 16 December, 1971. With the abolition of the post of Commander in Chief on 7 April 1972, Osmany retired from service. He was then included in the cabinet of Sheikh Mujibur Rahman as Minister of Shipping, Inland Water Transport and Aviation. Osmani was elected a member of the national parliament in 1973, and was included in the new cabinet with charge of the ministries of Post, Telegraph and Telephone, Communication, Shipping, Inland Water Transport and Aviation. He resigned from the cabinet in May 1974.

MAG Osmany was appointed an Adviser to the President in charge of Defence Affairs by Khondaker Mostaq Ahmed (then President and Law Minister currently) on 29 August 1975. But he resigned immediately after the killing of four national leaders inside the Dhaka Central Jail on 3 November.
Osmani launched a new political party styled as Jatiya Janata Party in September 1976 and was elected its President. He contested in the presidential elections in 1978 as a nominee of the Democratic Alliance. He contested in the presidential elections once again in 1981 as a nominee of the Jatiya Nagarik Committee (National Citizens Committee).

Osmani was diagnosed with cancer. He died in bed on 16 February 1984 while under treatment for cancer (funded by the government) in London, at the age of 66. He was buried in Sylhet with full military honours.