One of the defining patterns of English life in which young people move from small towns with limited prospects to bigger cities to seek their fortune is in dramatic decline, research has revealed.

More young people are getting stuck where they grew up or went to university because they cannot afford rents in places where they can earn more money, according to the Resolution Foundation thinktank. It found the number of people aged 25 to 34 starting a new job and moving home in the last year had fallen 40% over the last two decades.

Whereas previous generations were able to move to big cities such as London and Manchester or regional hubs like Leeds and Bristol to develop their careers, the current millennial generation is enduring a slump in mobility caused by rising rents, which can wipe out the financial gains of a move.

Even moves over short distances were barely worth making, the data showed. A person on average earnings in Scarborough paying average rent would have been 29% better off if they had moved to Leeds in 1997 and paid average rent and earned average money. In 2018, rising rents and stagnant wages means the benefit after taking into account rent was just 4%.

As part of a government drive to help, West Somerset was chosen as an “Opportunity Area”, one of 12 parts of the country receiving government funding to improve social mobility. It will get a share of a £60m pot.

One catalyst for change flagged up in the report is the building of the Hinkley Point C nuclear power station, which is creating thousands of jobs in Somerset.

But the need to improve education is central. Just over half the children in West Somerset achieved a good level of development at the end of reception in 2015-16, compared with almost 70% nationally.

A significantly lower proportion of pupils from West Somerset go on to attend higher education compared to elsewhere. Only 26% of 16- to 18-year-olds who took A-levels or other level 3 qualifications in the 2013-14 academic year attended university the following year, compared with 38% in Somerset as a whole and 49% nationally.

Susan Clowes, the manager at the independent advice centre in Minehead, tries to point people in the right direction. “I feel the population is dispirited. It’s almost as if life is done to them. They just grab what they can. We deal with a lot of people here who are not profligate, who spend every penny wisely and still struggle.”

Frances Nicholson, the Somerset county council cabinet member for children and families, said solving the issue of social mobility was crucial. “The population is unbalanced,” he said. “If we don’t address it eventually there will be no society, no community. It’s as stark as that.”

Britain risks becoming more divided unless there is a renewed effort to reduce the gap between the “haves and have-nots”, the influential Social Mobility Commission has warned.

In a damning report the commission found two decades of government efforts had failed to deliver enough progress and urged ministers to adopt new approaches to tackle the problems in British society.

The commission’s chairman Alan Milburn warned “whole tracts of Britain feel left behind” in “volatile and uncertain times”.

In an analysis of efforts to bridge the gap between rich and poor under Tony Blair, Gordon Brown, David Cameron and Theresa May, the commission found failings at every stage of a person’s life.

It follows warnings by Ofsted about a concentration of under-performing schools in the north and midlands, and calls into question general rhetoric over a divide between the north and south.

In the social mobility index drawn up by the commission, West Somerset is the area ranked as the worst, closely followed by Norwich in East Anglia, Wychavon in the West Midlands and Corby in Northamptonshire.

Of the 10 worst areas for social mobility, three are in the east of England, three are in the east midlands and two are in the north west. The report also shows coastal areas like Blackpool, Great Yarmouth and Minehead and industrial towns, including Mansfield and Stoke, are “becoming entrenched social mobility cold spots”.

On the other end of the scale, all of the top 10 boroughs for social mobility are in London, with the report showing a “more concentrated divide” in life chances between London and its commuter belt and the rest of the country.

The report says: “A key factor in the dominance of these areas – especially those in Greater London – is the strong educational outcomes of disadvantaged young people at primary and secondary school and the relatively high chances they have of progressing to university.”

The report goes on to explain that a large number of social mobility cold-spots are formerly-prosperous seaside resorts, which were built on a booming tourist trade, and have “struggled in the last few decades due to increased competition from Mediterranean resorts”.

These disadvantages are “accentuated by poor transport links to England’s main urban centres”, the report says.

Former Labour MP and minister Alan Milburn, who chairs the commission, said the report “laid bare the local lottery in social mobility”, and got “beneath the surface of a crude north/south divide and calls into question some of the conventional wisdom about where disadvantage is now located”.

He said: “I hope the government will put itself at the head of a new national drive to ensure that in future progress in life depends on aptitude and ability, not background and birth: on where people aspire to get to, not where they have come from. This report suggests that is long overdue.”