Your question is difficult to answer without a lot more details about the situation. For example, when you say that a pastor is taking the church away from the Bible, that can mean a couple of different things:
1. He isn't preaching from the Bible as much as you feel he should (e.g. he is preaching what I call "fluff sermons" wherein he has some moral message or story and is sprinkling in some Bible verses now and then to make it sound "churchy"). While I do not agree at all with that type of teaching, it would be difficult for a church board to remove a pastor based on that alone. In that case, the pastor should simply be challenged to get back to preaching from God's Word and, if he does not agree, it may be time for you to find a church where the pastor does preach from God's Word more in line with what you expect. (More on that further below.)
Along those lines, you should always be sure that you are reading God's Word on your own and being heavily involved in a small group Bible study, where you can be challenged. Sunday morning sermons should be more like "desert", though arguable many people do not read the Word of God as they should and so pastors should be much better about giving "solid food" during that time too.
I personally believe that if there is an error one way or the other, then pastors should attempt to bring those that do not read God's Word up, rather than water down the gospel and risk bringing others down or otherwise not challenge people enough with God's Word. It's a difficult balance, but really the only way a church is going to grow spiritually (which is not always the same as growing numerically, though they are not mutually exclusive) is if pastors take their leadership roles very seriously, quit worrying about "numbers" and preach the Word of God!
Of equal importance, if a church teaches people anything contrary to the Bible, the leader thereof can, in a sense, be considered a "false prophet". Jesus warns us that many false Christs and false prophets will try to lead us astray, especially in the end times (Matthew 24:5; Matthew 24:11; Matthew 24:24; Mark 13:6; and Mark 13:22).
If your pastor is preaching things against God's Word, then you should try to work within your church to get that corrected. Ideally you can find some others that agree with you, and then talk to some people on the church board so that you can discuss your mutual concerns with your pastor. The manner in which you do all that is greatly based on what you discern to be the pastor's intentions (is he doing it knowingly or out of ignorance of the Word?) Neither case is good, but it may be that he could be set back on track in a gentle, loving manner.
Yet I must also give a word of caution, as there are so many issues involved with "church politics". For example, the board members may be friends of the pastor, and/or perhaps they're selected by the pastor (in which case they can often be of the personality type that he knows won't challenge him). Many other dynamics can be involved too, such as how long he has been there, what his motives are, how long you have attended the church, and even the size of the church can impact how much influence you may or may not have as one voice alone.
You must be careful not to ruin a pastor's career if it is merely that you do not agree with his preaching style. There may be a point where you just have to "move on" and gracefully find another church that is better for you. Undoubtedly there are many, many different types of churches because not everyone agrees 100% on everything, people's tastes differ, etc. Paul explains in Romans chapter 14 that we must be careful not to argue over "disputable matters."
A resource I can highly recommend is The Church Awakening by Chuck Swindoll. Since you are obviously someone interested in what a Church should and should not be, you will very likely find it personally helpful. Once you read it, you can, if appropriate, pass it along to others (perhaps you will feel moved to buy a copy for your church board members if you can — I think every church leader should read this book!).
The book does a great job of challenging today's churches to renew their focus and ministries. It addresses where many churches are falling short these days, as they try to appease the crowds and fail to preach God's Word.
One of my favorite sections begins on page 88 and continues for several pages. In essence, Chuck mentions 2 Timothy 4:2 wherein it reads, "Preach the Word; be prepared in season and out of season; correct, rebuke and encourage--with great patience and careful instruction." Chuck makes the point that everything else falls short of preaching the Word of God in the pulpit. He goes on to explain, quite accurately, that way too many churches are failing their mission to teach from the Bible. Rather, they have resorted to marketing strategies and a harmful fixation on entertaining, being "politically correct", being "relevant", and being "in touch".
Chuck makes another good point with the verses found in Matthew 28:19-20 where we read, "Therefore go and make disciples of all nations, baptizing them in the name of the Father and of the Son and of the Holy Spirit, and teaching them to obey everything I have commanded you. And surely I am with you always, to the very end of the age." He accurately points out that the command does not just say we evangelize and then the mission is over. Rather, we are to teach people to obey everything that God commanded. A disciple is not one that has a shallow understanding of God's Word and while everyone must start at the beginning, at some point solid spiritual food must be provided within the Church.
I would like to end with these important words from Hebrews 10:25, “Let us not give up meeting together, as some are in the habit of doing, but let us encourage one another — and all the more as you see the Day approaching.”
Be sure to pray about all that you do so that whatever you do will be held blameless and righteous in the eyes of the Lord.
Pray That You May Escape - discusses the end times, to include Church apostasy