Gunmen from the Islamist group ordered non-Muslim passengers off the Nairobi-bound bus before dawn outside the town of Mandera, near Kenya's border with Somalia and Ethiopia.
Those who could not recite passages from the Koran were shot in the head.
One passenger who was spared, Ahmed Mahat, told the BBC that there were more than 60 people on board the bus when it was attacked by about 10 heavily armed men speaking Somali.
He said the driver tried to speed away, but the bus became stuck in mud.
"When we got down, passengers were separated according to Somali and non-Somalis," Mr Mahat told the BBC.
"The non-Somalis were ordered to read some verses of the holy Koran, and those who failed to read were ordered to lie down. One by one they were shot in the head at point blank range."
In an online statement, al-Shabaab said the attack was retribution for raids by security forces on mosques in the Kenyan coastal city of Mombasa earlier this week.
Kenya's inspector general of police, David Kimaiyo, told a news conference that most of the dead were Christians.
"The attackers, who were heavily armed, later fled towards the border between Kenya and Somalia," Kimaiyo said. "I have already ordered for a combined security team to be able to carry out a major operation and a crack down of the attackers."
On its Twitter feed, Kenya's interior ministry said it had launched "offensive action on the criminal gang", attacking their camp using helicopters and jets.
It said many were killed in the operation.
"I want to assure the people and the residents of Mandera that security will be enhanced and that more officers will be deployed in that particular area," Kimaiya said.
The Somali militant group has carried out a number of attacks in Kenya in recent years, the most prominent of which was a siege on a Nairobi shopping mall in September 2013, in which more than 60 people were killed.
National Security Council spokesperson Bernadette Meehan said the U.S. condemned the attack in the strongest terms.
"We extend our deepest condolences to the families and loved ones of the 28 individuals killed," she said.
"The United States stands with our Kenyan partners in the effort to counter the threat of terrorism and affirms our ongoing commitment to working with all Kenyans to combat these atrocities" she said.