Australian Prime Minister Tony Abbott has rejected a proposal from Aboriginal leaders that would allow them to draw up the framework for constitutional reform.

Several respected Aboriginal leaders have been in talks with Abbott about being recognized in the constitution as Australia's first people, but the prime minister on Tuesday rejected the proposal, claiming it would be divisive.

Abbott told the leaders there was a "risk" that it would be " unlikely to receive general support" if the referendum was purely led by Indigenous Australians.

"I am in favor of building consensus, but strongly believe this should be a national consensus in favor of a particular form of recognition rather than simply an Indigenous one," he told the leaders in a letter on Tuesday.

"The risk with an Indigenous-only, or even an Indigenous-first process is that it might produce something akin to a log of claims that is unlikely to receive general support."

Kirstie Parker, co-chair of the National Congress of Australia' s First Peoples, told the Australian Broadcasting Corporation (ABC) that Aboriginal leaders were dismayed by Abbott's comments.

She said they would not be "giving up" on their quest for recognition, saying Indigenous input was crucial was to achieving wide-spread acceptance.

"The Prime Minister has talked about a more general process that we don't think is satisfactory, will make it difficult if not impossible for our people to arrive at some sort of broad position in any direction," Parker said on Tuesday.

"What (Indigenous-only consultation) hopefully will lead to is an Aboriginal and Torres Strait Islander broad position.

"We are not giving up on the request that we've made to government on this."

This is the second time in a month that Abbott has faced backlash from Indigenous leaders. In July, he was accused of " pretending to listen" to leaders who were discussing the road to Aboriginal constitutional recognition.

Parker, at the time, said Abbott was disinterested in agreeing to a way forward.

"It remains to be seen how much real listening was done," she said on July 7 at a referendum in Canberra.

Opposition leader Bill Shorten said the government needed to keep open dialogue between the leaders to achieve the landmark recognition.

"We need to remain flexible on how we achieve constitutional recognition," he told the ABC on Tuesday.

"Part of that is ensuring Indigenous Australians have their views genuinely heard."

There is currently no mention of Aboriginal and Torres Strait Islander people in the constitution, and no recognition they were the first Australians.