The International Committee of the Red Cross said it took part in the humanitarian mission Wednesday in western Choco department. There were no other immediate details.
Duque suspended peace talks with the National Liberation Army, or ELN, upon taking office a month ago, insisting that for negotiations to resume the rebels must stop all criminal activity, including kidnappings and attacks on oil pipelines.
The conservative leader showed little interest in softening his position as news of the release emerged. While celebrating that the captives would soon be reunited with loved ones, he called his predecessor's decision to negotiate with rebels even as they continued to carry out attacks a "historic mistake" that he would not repeat.
"If there is a genuine desire for peace, the government will be there," Duque said in a speech in the southern city of Pasto. But "if we want to build peace with this armed group they need to begin by stating clearly that they will suspend all criminal activities."
The three police officers, army soldier and two civilian contractors were taken captive more than a month ago when the small boat they were traveling on was intercepted by a guerrillas belonging to the ELN who were patrolling a remote river in western Colombia.
Authorities said the group was dressed in civilian clothing and returning a rural zone where members of another rebel group, the Revolutionary Armed Forces of Colombia, had demobilized as part of a 2016 peace deal.
The government maintains that the ELN is holding another 10 civilian hostages. The rebels haven't confirmed that number.
The ELN has said it will only cease attacks if the government agrees to a bilateral ceasefire.
The ELN was created in 1964 by a group of Catholic priests and Marxist activists inspired by the Cuban revolution. It has approximately 1,500 fighters and it became the country's last remaining guerrilla group after the much-larger FARC disbanded as part of the peace deal with Duque's predecessor.
Former President Juan Manuel Santos began peace talks with the ELN that have yielded few commitments so far. The talks were suspended at the beginning of this year after the rebel group broke a bilateral ceasefire that had lasted for three months, but they resumed in May even while fighting between the government and the ELN continued.
Copyright 2019 The Associated Press. All rights reserved. This material may not be published, broadcast, rewritten or redistributed.