Maude has served as minister since 2010 and is due to step down as an MP in May.
He was keen to highlight his achievements, such as setting up the Government Digital Service (GDS) and GOV.UK, opening up more data and introducing new IT procurement rules during a speech at techUK today.
However he warned: "We've started to deal with our legacy but we're not out of it yet. We still have huge amounts of legacy IT".
The importance of modernising and replacing legacy so it can cope with real-time, 21st century technology has been discussed for years in Whitehall. The Department for Work and Pensions' legacy systems still run off VME mainframes installed in 1974.
HM Revenue & Customs is currently readying to replace its £10 billion 'Aspire' contract with Capgemini in 2017, and the Home Office is preparing for the end of deals with Atos and Fujitsu next year.
Maude said the government saved £91 million last year thanks to "militantly enforced" new Cabinet Office guidelines limiting the size, length and scope of IT deals in Whitehall.
The rules, which were introduced in January 2014, say departments should not award IT contracts worth over £100 million, should never automatically extend contracts, should limit hosting contracts to a maximum of two years and ensure they use different firms for systems integration and IT services.
Maude claimed at, if enforced, new IT 'red lines' could save up to £20 billion by 2020 as longstanding, expensive IT contracts finish across Whitehall departments.
However just over a year since they were introduced, it is unclear if departments are following the new guidelines. For example, the Department for Work and Pensions extended a hosting deal with HP for three years citing 'technical reasons' last month.
Maude admitted "we're only just getting started" on improving government IT but said "the era of bespoke solutions commissioned at extortionate costs by departments is definitively over".
He described the 'Government as a Platform' agenda of building public services on a common technology platform as "the modern equivalent to building the Victorian sewer network or the National Grid. In future public services will need to be fully integrated."
Image: iStock/lorenzoantonucci