Yes, one can do very limited performance testing using Selenium webdriver, but it would be similar to say using a tea spoon to cook meal. Tea spoon is meant for a different purpose, maybe measuring sugar and stirring coffee/tea , but it can even be used for achieving other goals. The only thing is it would not suit one’s exact needs for the other thing.
There are many open source tools to do performance testing and it is advisable to use them.
One sexy use of Selenium webdriver in conjunction with performance testing tools such as jmeter is running your web app’s selenium tests during performance test. There are some tricky ways to achieve it and it gives pretty accurate results. For e.g. run your performance test generating normal to heavy load and simultaneously start selenium test to get parameters such as page load time etc to understand actual user experience of the system during such phases.