[web2py] building a decision tree in web2py

2012-05-28 Thread Anj
hi! I have python codes for building an id3 decision tree and they`re 
already running. However, the output decision tree is just text-based. So, 
I decided to use web2py to give it a more pleasant look. Now, my problem is 
how do i integrate these codes in web2py. I mean, where will I insert these 
codes especially the computations of entropy and information gain after I 
upload my dataset? How will I make an OOP into an MVC? 

By the way, I have already created the GUI for this application but still 
lack a UI for displaying the output decision tree.

Here`s the scenario:



that`s the Home Page. When I click the ID3 menu, it will redirect here:



The I will upload a file as my dataset. The dataset is to be computed and 
analyzed so that the program will display the output. 

Please help me. Thanks!

"""
This module holds functions that are responsible for creating a new
decision tree and for using the tree for data classification.
"""

def majority_value(data, target_attr):
"""
Creates a list of all values in the target attribute for each record
in the data list object, and returns the value that appears in this list
the most frequently.
"""
data = data[:]
return most_frequent([record[target_attr] for record in data])

def most_frequent(lst):
"""
Returns the item that appears most frequently in the given list.
"""
lst = lst[:]
highest_freq = 0
most_freq = None

for val in unique(lst):
if lst.count(val) > highest_freq:
most_freq = val
highest_freq = lst.count(val)

return most_freq

def unique(lst):
"""
Returns a list made up of the unique values found in lst.  i.e., it
removes the redundant values in lst.
"""
lst = lst[:]
unique_lst = []

# Cycle through the list and add each value to the unique list only once.
for item in lst:
if unique_lst.count(item) <= 0:
unique_lst.append(item)

# Return the list with all redundant values removed.
return unique_lst

def get_values(data, attr):
"""
Creates a list of values in the chosen attribute for each record in data,
prunes out all of the redundant values, and return the list.  
"""
data = data[:]
return unique([record[attr] for record in data])

def choose_attribute(data, attributes, target_attr, fitness):
"""
Cycles through all the attributes and returns the attribute with the
highest information gain (or lowest entropy).
"""
data = data[:]
best_gain = 0.0
mylist = ['Easy', 'Average', 'Difficult']
best_attr = temp1 = None
newlist = []
  #  i -=1
for attr in attributes:
gain = fitness(data, attr, target_attr)
if (gain >= best_gain and attr != target_attr):
best_gain = gain
best_attr = attr
temp = best_attr
   # print (best_attr)
return best_attr

def get_examples(data, attr, value):
"""
Returns a list of all the records in  with the value of 
matching the given value.
"""
data = data[:]
rtn_lst = []

if not data:
return rtn_lst
else:
record = data.pop()
if record[attr] == value:
rtn_lst.append(record)
rtn_lst.extend(get_examples(data, attr, value))
return rtn_lst
else:
rtn_lst.extend(get_examples(data, attr, value))
return rtn_lst

def get_classification(record, tree):
"""
This function recursively traverses the decision tree and returns a
classification for the given record.
"""
# If the current node is a string, then we've reached a leaf node and
# we can return it as our answer
if type(tree) == type("string"):
return tree

# Traverse the tree further until a leaf node is found.
else:
attr = tree.keys()[0]
t = tree[attr][record[attr]]
return get_classification(record, t)

def classify(tree, data):
"""
Returns a list of classifications for each of the records in the data
list as determined by the given decision tree.
"""
data = data[:]
classification = []

for record in data:
classification.append(get_classification(record, tree))

return classification

def create_decision_tree(data, attributes, target_attr, fitness_func):
"""
Returns a new decision tree based on the examples given.
"""
data = data[:]
vals = [record[target_attr] for record in data]
default = majority_value(data, target_attr)

# If the dataset is empty or the attributes list is empty, return the
# default value. When checking the attributes list for emptiness, we
# need to subtract 1 to account for the target attribute.
if no

[web2py] Re: building a decision tree in web2py

2012-05-28 Thread Anj
I have already done that. i just need to revise those that are in the 
test.py to conform to web2py structure

Noong Lunes, Mayo 28 2012 03:02:47 UTC-12, si Anthony ay sumulat:
>
> Put dtree.py and id3.py in /web2py/application/yourapp/modules/. See 
> http://web2py.com/books/default/chapter/29/4#Accessing-the-API-from-Python-modules
> .
>
> Anthony
>
> On Sunday, May 27, 2012 11:00:34 PM UTC-4, Anj wrote:
>>
>> hi! I have python codes for building an id3 decision tree and they`re 
>> already running. However, the output decision tree is just text-based. So, 
>> I decided to use web2py to give it a more pleasant look. Now, my problem is 
>> how do i integrate these codes in web2py. I mean, where will I insert these 
>> codes especially the computations of entropy and information gain after I 
>> upload my dataset? How will I make an OOP into an MVC? 
>>
>> By the way, I have already created the GUI for this application but still 
>> lack a UI for displaying the output decision tree.
>>
>> Here`s the scenario:
>>
>>
>> <https://lh3.googleusercontent.com/-FfLOuWzGqFI/T8Lprknyu9I/AaI/RDkVYOosaT8/s1600/id31.JPG>
>>
>> that`s the Home Page. When I click the ID3 menu, it will redirect here:
>>
>>
>> <https://lh3.googleusercontent.com/-brRNheqzg04/T8Lp8L1oi3I/AaQ/oRg3xGw73_0/s1600/id32.JPG>
>>
>> The I will upload a file as my dataset. The dataset is to be computed and 
>> analyzed so that the program will display the output. 
>>
>> Please help me. Thanks!
>>
>>

[web2py] Re: building a decision tree in web2py

2012-05-28 Thread Anj
hi! the problem is not really on the display itself but on how shall i 
build the decision tree using those functions in test.py so that i can 
display the output. you see, i need to go through the contents of the 
dataset file before i can have the dtree output.

Noong Lunes, Mayo 28 2012 02:52:42 UTC-12, si Andrew ay sumulat:
>
> Hi,
> Is your specific question how to visually display a decision tree in a 
> web2py application ?
>
> I have been playing around with D3 (Data Driven Documents)  see d3js.org, 
>  which I think is the best thing since sliced bread !
> Have a look at this example for some ideas / possibilities.   
> http://bl.ocks.org/999346, or this one http://bl.ocks.org/1249394
>
> I have built a few simple proof of concept apps with web2py by building 
> controllers that return json files for the thing you are visualising.  
> However, in the words of Ward Cunningham talking about D3 "There is a 
> learning curve there", but he also said looking into it's code "... The 
> most rewarding work I've done this year"
>
> You could also use Graphviz, which I've also used with web2py, but I'm now 
> trying to get my graphviz diagrams (static diagrams) to come out more 
> dynamically with d3.
> Have a look and let me know what you think.
>
> Andrew W
>
> On Monday, May 28, 2012 3:00:34 PM UTC+12, Anj wrote:
>>
>> hi! I have python codes for building an id3 decision tree and they`re 
>> already running. However, the output decision tree is just text-based. So, 
>> I decided to use web2py to give it a more pleasant look. Now, my problem is 
>> how do i integrate these codes in web2py. I mean, where will I insert these 
>> codes especially the computations of entropy and information gain after I 
>> upload my dataset? How will I make an OOP into an MVC? 
>>
>> By the way, I have already created the GUI for this application but still 
>> lack a UI for displaying the output decision tree.
>>
>> Here`s the scenario:
>>
>>
>> <https://lh3.googleusercontent.com/-FfLOuWzGqFI/T8Lprknyu9I/AaI/RDkVYOosaT8/s1600/id31.JPG>
>>
>> that`s the Home Page. When I click the ID3 menu, it will redirect here:
>>
>>
>> <https://lh3.googleusercontent.com/-brRNheqzg04/T8Lp8L1oi3I/AaQ/oRg3xGw73_0/s1600/id32.JPG>
>>
>> The I will upload a file as my dataset. The dataset is to be computed and 
>> analyzed so that the program will display the output. 
>>
>> Please help me. Thanks!
>>
>>

[web2py] Re: building a decision tree in web2py

2012-05-28 Thread Anj
hi! i`ve seen your work and it`s great! will you let me use your code if 
ever i decide to make my display look like that?

Noong Lunes, Mayo 28 2012 02:52:42 UTC-12, si Andrew ay sumulat:
>
> Hi,
> Is your specific question how to visually display a decision tree in a 
> web2py application ?
>
> I have been playing around with D3 (Data Driven Documents)  see d3js.org, 
>  which I think is the best thing since sliced bread !
> Have a look at this example for some ideas / possibilities.   
> http://bl.ocks.org/999346, or this one http://bl.ocks.org/1249394
>
> I have built a few simple proof of concept apps with web2py by building 
> controllers that return json files for the thing you are visualising.  
> However, in the words of Ward Cunningham talking about D3 "There is a 
> learning curve there", but he also said looking into it's code "... The 
> most rewarding work I've done this year"
>
> You could also use Graphviz, which I've also used with web2py, but I'm now 
> trying to get my graphviz diagrams (static diagrams) to come out more 
> dynamically with d3.
> Have a look and let me know what you think.
>
> Andrew W
>
> On Monday, May 28, 2012 3:00:34 PM UTC+12, Anj wrote:
>>
>> hi! I have python codes for building an id3 decision tree and they`re 
>> already running. However, the output decision tree is just text-based. So, 
>> I decided to use web2py to give it a more pleasant look. Now, my problem is 
>> how do i integrate these codes in web2py. I mean, where will I insert these 
>> codes especially the computations of entropy and information gain after I 
>> upload my dataset? How will I make an OOP into an MVC? 
>>
>> By the way, I have already created the GUI for this application but still 
>> lack a UI for displaying the output decision tree.
>>
>> Here`s the scenario:
>>
>>
>> <https://lh3.googleusercontent.com/-FfLOuWzGqFI/T8Lprknyu9I/AaI/RDkVYOosaT8/s1600/id31.JPG>
>>
>> that`s the Home Page. When I click the ID3 menu, it will redirect here:
>>
>>
>> <https://lh3.googleusercontent.com/-brRNheqzg04/T8Lp8L1oi3I/AaQ/oRg3xGw73_0/s1600/id32.JPG>
>>
>> The I will upload a file as my dataset. The dataset is to be computed and 
>> analyzed so that the program will display the output. 
>>
>> Please help me. Thanks!
>>
>>