The elders are accountable to the church, to each other, and most of all to the Lord. When they are healthy their love for all the saints under their care should cause them to continually seek the Lord's leading and speaking. At the same time their fear of God should prevent them from being careless or impure in their oversight over the saints. Nevertheless, people always have the capacity to fall into the snare of the evil one. Very often the devil will tempt anyone in a position of leadership with pride (1 Tim. 3:6). This can be a problem no matter how much you have grown in the Lord. If your life and ministry is truly spiritual then the Lord may bless it greatly. The devil will surely tempt you to turn your gaze from the Lord to the success of your work. That is when the question of accountability in leadership becomes very relevant.
The eldership should always be corporate, that is, there should be more than one elder. Thus, if one elder stumbles, hopefully the others can help him to recover. Also, the elders need to be open to the fellowship of all the saints. Every member should feel free to present their concerns to the elders even if it is about one of the other elders. If this is done with a proper spirit out of a love and concern for the church then the elders need to respect it and bring this matter to the Lord.
Eventually, however, there are times when all these ways to resolve a problem in the eldership fail. Then, either the other elders or some of the rest of the saints, need to bring this matter to the apostle who is ministering to the churches in the area. Eventually, if all else fails, he may need to remove one or more of the elders from their position.