If you are looking for open source alternative of Matlab then try gnu octave . according to octave official site :
GNU Octave is a high-level interpreted language, primarily intended for numerical computations. It provides capabilities for the numerical solution of linear and nonlinear problems, and for performing other numerical experiments. It also provides extensive graphics capabilities for data visualization and manipulation. Octave is normally used through its interactive command line interface, but it can also be used to write non-interactive programs. The Octave language is quite similar to Matlab so that most programs are easily portable.
Installing Octave on Mac
follow these simple steps to get and running octave on your mac osx ( i am using mavericks 10.9 in this case ) .
Lest first install some requirements .
and install xcode command line tools
then open terminal and type
ruby -e "$(curl -fsSL https://raw.github.com/Homebrew/homebrew/go/install)"
now update and tap homebrew science
brew update brew tap homebrew/science
now install gcc and automake ( just make sure its compile )
brew tap homebrew/versions brew install gcc49 brew install gfortran brew install octave
if you having trouble in mavericks use that patch ( it worked for me )
brew install https://raw.github.com/Homebrew/homebrew-science/3c3fe3baaf926437f750f65456769c124d6be8e1/octave.rb --env=std