What is Jupyter Notebook?

I have been exploring Python and R, and stumbled on a dashboard called Jupyter Notebook.

The Jupyter Notebook is an open-source web application that allows you to create and share documents that contain live code, equations, visualizations and narrative text. Uses include: data cleaning and transformation, numerical simulation, statistical modeling, data visualization, machine learning, and much more.

From their website – https://jupyter.org

I have found that as an instructor, I can share files with my students and they can collaborate together and with me. This is done through a program called Anaconda which acts as a server. Anaconda is a free package manager that contains over 1,000 open source packages with community support and can be used on Windows MacOS or Linux.

Steps to install:

  1. Navigate to https://docs.anaconda.com/anaconda/install/ and download/install the program * there are step-by-step direction on the site
  2. Make sure you have Python with PIP already installed.
  3. If you have Python 3 installed, use PIP to install it

python3 -m pip install –upgrade pip

python 3 -m pip install jupyter

Run the following command at the terminal or command prompt.

jupyter notebook

I created a new folder called My Notebooks where I will store everything. You can create a new folder by either selecting New in the upper right hand corner of the notebook or create and name a new folder on your computer and navigate to it.

Let’s get started and create a Python program.

  1. Select New and select Python 3
  2. Type or copy/paste the following code:

# This program adds two numbers

num1 = 2.75

num2 = 3.50

# Add two numbers

sum = float(num1) + float(num2)

# Display the sum

print(‘The sum of {0} and {1} is {2}.format(num1, num2, sum))

Click Run and you will get the following output:

The sum of 2.75 and 3.50 is 6.25

For a video tutorial see Corey Schafer’s video on youtube: https://www.youtube.com/watch?v=HW29067qVWk

The Digital Dark Age – What is it and how do we reduce its impact?

“We do not have a systematic way of making sure that the digital information that we create today will still be usable 100 years from now” – Vint Cerf. 

It used to be that paper receipts were kept for the purposes of tracking expenses at home and in the workplace. However, overtime, ink fades making the receipt illegible. When information is printed onto paper using ink, it is subject to degradation over time or can be damaged from environmental factors like water or fire. Technology allows people to store their receipts including valuable documents digitally which makes them easier to categorize and search, and easier to access later. Decades ago, when people wanted to take pictures of family, trips, and important moments, they would use a camera with film and they would have to have that film developed and printed. Today, cellphones can capture and edit the picture, share those images to social media and save them to a cloud storage – storing the images as jpg files. What happens when decades from now we are no longer able to read jpg files? What about documents created in the 80’s or 90’s that used software that is no longer available? How do we access the information on those documents?

The pioneer of the Internet, Vint Cerf stated the there is a need to develop technologies, tools, financial resources, shared responsibilities to prevent the loss of data (bits).  Data will continue to be lost due to not only deterioration of hardware but also to software that may not be available. There is a fear that there is so much reliance on storing information in digital form that an entire generation’s data may be lost. One side of this issue argues that reliance on digital data including cloud computing, Internet of Things and big data will only increase the scale and scope of the darkness. The other side of this issue is that this reliance on digital forms will help to preserve the data and ensure there is always an ability to see it.

Big Data

Big data is a term that has been around for approximately two decades and describes the growth of data in terms of value, volume, velocity, veracity, variability and visualization. Big data produces substantial benefits and revenue, but many organizations are still learning what data is needed and therefore kept versus data that is useless and therefor discarded. These businesses have been told that storage is inexpensive so keeping the information for an infinite period of time is not harmful. It was estimated that 90% of global data in 2011 was created in the previous two years. For most organizations, information volumes double about every two years with masses of information snowballing along with the cost to store, secure and manage the data. Only about 15% of information created in most organizations has value over time (Ragan, 2014). One problem that industries are seeing with their data that is not already in the cloud is that they have collected a hodge-podge mass of data which is only accessible on-site using applications like Excel or some other program that may use xml files, and some of these applications are outdated making it harder to effectively use the data.

Born-digital Content

Born-digital content included published and licensed works like e-journals and e-books, software applications, video games, medical data, social media content, research data, electronic archives, and live feeds including RSS feeds – just to name a few. There is a challenge inherent to the born-digital content because user behaviors and expectations are changing. There is a need to grow and adapt quickly which is hindered by the need for systemic change and issues in protecting that data. The relationship between form, text and function must be considered with born-digital resources because physical and format are no longer as important as authenticity and origin (Neal, 2015).

Cloud Migration

Hardware that stores the data will not last forever – time, environment, and neglect affects its ability to still be readable. The hardware issue is that devices used to read that hardware may no longer be available.  The software issue is bitstream for word processing document without the correct application, software, operating system, and hardware environment to interpret and repacking the data is not possible. Digital content is at risk because of uncontrolled collection of data, accidental destruction of data, unauthorized tampering if data, lack of metadata and systems documentation. Media deterioration and technological uselessness (Whitt, 2017). A solution to this problem is to migrate all data to the cloud which will reduce IT costs while increasing productivity, scalability and flexibility.

The cloud has become a centralized place for storage and access. Cloud computing allows users to manage and analyze data collaboratively whether as infrastructure as a service [IaaS], platform as a service [PaaS] or software as a service [SaaS] (Whitt, 2017). Migrating current data into the cloud requires a strategy. Critical Success Factors [CSF] identified in literature include technological and organizational factors. Technological CSFs include reliability, interoperability, security and privacy, disaster recovery and network bandwidth. Organizational CSFs include management support, IT Training, users’ awareness, and degree of control (Alharthi, Alassafi, Walters & Wills, 2016).

Digital Preservation

Digital preservation is the ensuring full access and continued usability of data and digital information. Literature has also defined digital preservation as being a series of managed activities ensuring continued access to digital materials. A 1996 Task Force report defined long-term digital preservation and used the term digital archiving. Archiving includes the concept of assured access to content, separate from it preservation. Access is seen as continuous and ongoing usability of a digital resource, retaining authenticity, accuracy and functionality.  There are three community documents formalized the digital preservation practice and they include Open Archival Information System [OAIS] Reference Model, the Trusted Digital Repository Report and the PAIMAS Standard. The InterPARES or the International Research on Permanent Authentic Records in Electronic Systems was launched in 1999 as a collaborative research which focused on long-term preservation of authentic digital materials (Whitt, 2017).

The objective in digital preservation is to preserve access to the digital content rather than the physical object. There are three types of technological obsolescence including hardware issues where there may be no compatible device, software where there is no backward compatibility, and media deterioration which includes deterioration of physical storage medium. As technology evolves, software and hardware will inevitably become obsolete.  There is an irony in data that in the analog word, records can survive for a long time as long as they do not get wet, eaten by rodents or burned. However, bit patters are logically unstable with their local structure changing each time the file is opened (Whitt, 2017).    

Conclusion

Vint Cerf in his interview on CBS Sunday Morning (2018), stated he is working on what he calls digital vellum, but will this really help? How much of our data should be saved? Until that happens, how do we preserve our data from being lost? Some short-term answers are to save everything in the cloud, on multiple types of media like optical and flash drives. Some would argue to just save everything as a pdf file because we will always be able to read those even if Adobe no longer exists, there is enough freeware available that can read pdf files we should be okay. We need to start thinking about 20, 30, 40 years from now. There is no one solution that will definitely be available. As technology continues to evolve at an alarming rate, so does our ability to predict the future of what will be around tomorrow.

 References

Alharthi, A, Alassafi, M. O., Walters, R. J. & Wills, G. B. (2016). An exploratory study for investigating the critical success factors for cloud migration in the Saudi Arabian higher education context. Telematics and Informatics, 34. 664-678.

CBS Sunday-Moring. (2018). Vint Cerf on the prospect of a “digital dark age”. [Video File]. Retrieved from https://www.youtube.com/watch?v=uvob8wCMhLo

Neal, J. G. (2015). Preserving the born-digital record. American Libraries Magazine.

Ragan, C. R. (2014). Avoiding the digital dark age and remaining competitive. NACD Directorship.

Whitt, R. S. (2017). Through a glass, darkly – Technical, policy, and financial actions to avert the coming digital dark ages. Santa Clara High Technology Law Journal, 33(2). 117-229.

MOOCS: Big Data, Security, and Ethics

Big data is the use and collection of large amounts of data and is considered a game changer with the ability to change the way businesses operate in industries around the world.   The study of customer activity and behavior has become essential to learn about users’ experiences and is becoming a key decision-making tool. Education has jumped into the big data pool to better understand its students and their behaviors, providing a more efficient learning environment and identifying students at risk earlier in their academic progress.

Benefits of Big Data

Analyzing big data includes the integration of data from multiple sources including phone, email, and instant message and helps the organization to better understand their customers’ needs, wants, and problems quicker than doing a survey or waiting for an unhappy customer to contact them. Big data can be analyzed in real-time to detect fraudulent activities, and notify clients of potential issues promptly (Lee, 2017). Higher educational institutions who offer online classes can track attendance and engagement in the classroom. This allows administrators and faculty to proactively help that student be more successful or help them overcome any challenges they may be facing. The Open University uses learning analytics to proactively identify interventions which support students in achieving their study goals. Data is systematically collected and analyzed to identify risk factors across students. The data collected allows for students to categorized in varying levels of risk-bearing groups where a different intervention can be used to further help the student (Johnson, 2017). A good example of this would be to look at the amount of time a student spends participating in active learning activities like classroom live chat area and the amount of time that student spends in passive learning like watching a video or reading a lecture. It can be analyzed if students spend more time in one or the other may be more successful or may be at a high risk because little or no time is being spent doing either of those activities. 

Challenges of Privacy in Big Data

Data is increasing and expanding at a rapid rate. Big data in its definition is defined by its veracity, variability, complexity, and value, all adding to the challenges in protecting the privacy of users (Lee, 2017). In the context of higher education, if students know they will be treated differently based on their data, this could potentially impact their liberty and autonomy. The education institutions are using the data to mark students as risky are not trying to exclude students from opportunity but to intercede to help students succeed. Higher education tries to enhance a student’s autonomy and may implement data collection so they can improve at educating students and a better job of enhancing their autonomy. There is a potential trade-off between recognition or respect for the autonomy of students on one side and compassionate efforts to increase autonomy for groups of students on the other side (Scholes, 2016).

Recommendations

Protection of consumer privacy is a growing challenge in industries. Protection of privacy is sometimes counterproductive to both the organization and its customers as big data is a key tool to help the organization in improving its products, services, and decision-making. Firms and its customers need to find the right balance the use of personal data for services and privacy concerns. There is no one solution that works, no one-size-fits-all measure for privacy, but the balance depends on service type, customers served, data type, and regulatory environments (Lee, 2017).

References

Johnson, J. A. (2017). Ethics and justice in learning analytics. New Directions for Higher Education2017(179), 77–87. https://doi-org.proxy1.ncu.edu/10.1002/he.20245

Lee, I. (2017). Big data: Dimensions, evolution, impacts, and challenges. Business Horizons, 60. 293-303.

Scholes, V. (2016). The ethics of using learning analytics to categorize students on risk. Educational Technology Research & Development64(5), 939–955. https://doi-org.proxy1.ncu.edu/10.1007/s11423-016-9458-1

Driving Traffic

Photo by Pixabay on Pexels.com

We all strive to be the highway on the left with all of the traffic.  Unfortunately, most of us especially newbies are happy when we see even minimal traffic. While I am somewhat new to blogging, I am very familiar with the need and value of driving traffic to your site.  I currently have a Youtube channel with several hundred subscribers, a Linkedin profile with about the same amount and a Facebook profile of about 1,000 members. I created a video for my novel and provide a brief synopsis.  I also have a Twitter account with over 900 followers.

This book is the first in a series of 9 – so I am offering it for free to build a following for the series.  Once the first book is complete, I plan on publishing with Amazon – again, the main objective is to get people to read the book so I can get them hooked on the series. 

Here are a few of the things I have done so far:

Youtube

  • posted a video with the book synopsis and a link to the blog site

Twitter

  • tweeted with a link for Prologue
  • tweeted with a link for Chapter 1
  • continue building my twitter followers about 2-3x per week

Blog

  • Posted to my blog every couple of days
  • Continue writing/editing book so I can post 1 completed chapter each week

Facebook

  • Created a page for the novel and have posted all of the updates there