We understand your concern for your brother and we are here to pray and provide biblical guidance for you both. Firstly, let's remember what the Bible says about such experiences:
For though we walk in the flesh, we do not wage warfare according to the flesh; for the weapons of our warfare are not of the flesh, but mighty before God to the throwing down of strongholds, throwing down imaginations and every high thing that is exalted against the knowledge of God, and bringing every thought into captivity to the obedience of Christ 2 Corinthians 10:3-5 WEB.
Your brother's experiences may be a spiritual battle for his faith. It is important to remember that our struggle is not against flesh and blood, but against the rulers, against the authorities, against the powers of this dark world and against the spiritual forces of evil in the heavenly realms Ephesians 6:12 WEB.
Let us pray together:
Dear Heavenly Father,
We come before You in the name of Jesus Christ, lifting up our brother who is experiencing spiritual disturbances. We thank You for his faith in You and for drawing him back from his curiosity about Islam. We pray that You will guard his heart and mind with the peace that only You can give Philippians 4:7 WEB. We rebuke any spiritual attack on his mind and proclaim that he will not be deceived. We pray that You will give him discernment and strength to resist any such attacks in the future. Surround him with Your protection, Lord, and fill him with the knowledge of Your will in all spiritual wisdom and understanding, that he may walk worthy of You, Lord, to all pleasing, being fruitful in every good work, and increasing in the knowledge of God Colossians 1:9-10 WEB.
Father, we also pray for ###, that You will give them wisdom and guidance as they support their brother. Help them to be a source of encouragement and strength, pointing their brother always to You.
In Jesus' name, we pray. Amen.
Encourage your brother to remain steadfast in his faith in Jesus Christ. Remind him that it is only through Jesus that we have access to God:
Jesus said to him, I am the way, the truth, and the life. No one comes to the Father, except through me John 14:6 WEB.
And there is salvation in no one else, for there is no other name under heaven given among men, by which we must be saved Acts 4:12 WEB.
If your brother is willing, encourage him to join a strong, Bible-believing church where he can be nurtured and grow in his faith. It is important for him to be around fellow believers and ### - let us be his body who can support and encourage him.
Lastly, encourage your brother to put on the full armor of God so that he can take his stand against the devil's schemes Ephesians 6:11. The armor includes the belt of truth, the breastplate of righteousness, feet fitted with the readiness that comes from the gospel of peace, the shield of faith, the helmet of salvation, and the sword of the Spirit, which is the word of God Ephesians 6:14-17. This will help him to stand firm and be victorious in this spiritual battle.
Together, let us trust in the power of God to deliver your brother from these experiences and to strengthen his faith in Jesus Christ.
Prayer Focus: God, Thank You. Thank You. Thank You. God Thank You for loving me. Thank You for loving me, Jesus. God, I ask You in Jesus’ name bless me with everything that I stand in need of and everything You want me to have. Bless me to know You in truth, fall in love with You with all my heart, soul, mind, and strength and never fall out of love with You. Bless me to have an ever growing closer stronger, more intimate relationship with You. Bless me with the love, desire, strength, and the spirit of obedience to always delight myself in You, seek first Your kingdom, and Your righteousness. Help and strengthen me God to always respect and obey You. Bless me to trust You with all my heart, acknowledge You in all my ways, and lean not to my own understanding. 
