The Quran

وَيَومَ نَحشُرُهُم جَميعًا ثُمَّ نَقولُ لِلَّذينَ أَشرَكوا مَكانَكُم أَنتُم وَشُرَكاؤُكُم ۚ فَزَيَّلنا بَينَهُم ۖ وَقالَ شُرَكاؤُهُم ما كُنتُم إِيّانا تَعبُدونَ

(Saheeh International)

And [mention, O Muḥammad], the Day We will gather them all together – then We will say to those who associated others with Allāh, "[Remain in] your place, you and your 'partners.'"¹ Then We will separate them,² and their "partners" will say, "You did not used to worship us,³

Those they had associated with Allāh.
From the believers.
The inanimate objects, such as idols, will not have been aware of their worship of them. But those beings who consented to be worshipped will lie and deny it on the Day of Judgement.