A internal civil war has been my life for years as I've walked the razor wire separating breakers and builders.
I'd like to share my experiences presenting outside the security industry at developer conferences like Djangocon, JSConf (trackb), and RealtimeConf what it's like to be the lone security professional in a sea of builders, what it feels like, what they say, what they really care about, how not to speak to them, how to completely disrespect them, and how to fail miserably.
I’ll explain how React is moving towards a minimal API surface area. Instead of providing many framework features, React is trying to utilize patterns, paradigms and JavaScript language features to accomplish the same tasks that other frameworks have dedicated APIs for. Not all environments have these natively. We need to polyfill new an experimental features. How can we safely do that? Is it better to have a non-standard library?
Extending your Rails app with some Go, Scala, Elixir or node.js sound interesting to you? The first challenge to get there is to safely share the session between your apps. Crypto is hard… but that won't prevent us from looking into how Rails sessions work and how to share them across programming languages.
Python's scientific computing and data analysis ecosystem, built around NumPy, SciPy, Matplotlib, Pandas, and a host of other libraries, is a tremendous success. NumPy provides an array object, the array-oriented ufunc primitive, and standard practices for exposing and writing numerical libraries to Python all of which have assisted in making it a solid foundation for the community. Over time, however, it has become clear that there are some limitations of NumPy that are difficult to address via evolution from within. Notably, the way NumPy arrays are restricted to data with regularly strided memory structure on a single machine is not easy to change.
Blaze is a project being built with the goal of addressing these limitations, and becoming a foundation to grow Python's success in array-oriented computing long into the future. It consists of a small collection of libraries being built to generalize NumPy's notions of array, dtype, and ufuncs to be more extensible, and to represent data and computation that is distributed or does not fit in main memory.
Datashape is the array type system that describes the structure of data, including a specification of a grammar and set of basic types, and a library for working with them. LibDyND is an in-memory array programming library, written in C++ and exposed to Python to provide the local representation of memory supporting the datashape array types. BLZ is a chunked column-oriented persistence storage format for storing Blaze data, well-suited for out of core computations. Finally, the Blaze library ties these components together with a deferred execution graph and execution engine, which can analyze desired computations together with the location and size of input data, and carry out an execution plan in memory, out of core, or in a distributed fashion as is needed.
Spatial weights matrices, $W$, play an essential role in many spatial analysis tasks including measures of spatial association, regionalization, and spatial regression. A spatial weight $w_{i,j}$ represents potential interaction between each $i,j$ pair in a set of $n$ spatial units. The weights are generally defined as either binary $w_{i,j}=\{1,0\}$, depending on whether or not $i$ and $j$ are considered neighbors, or a continuous value reflecting some general distance relationship between $i$ and $j$. This work focuses on the case of binary weights using a contiguity criteria where $i$ and $j$ are rook neighbors when sharing an edge and queen neighbors when sharing a vertex.
Population of the $W$ is computationally expensive, requiring, in the naive case, $O(n^2)$ point or edge comparisons. To improve efficiency data decomposition techniques, in the form of regular grids and quad-trees, as well as spatial indexing techniques using r-trees have be utilized to reduce the total number of local point or edge comparisons. Unfortunately, these algorithms still scale quadratically. Recent research has also shown that even with the application of parallel processing techniques, the gridded decomposition method does not scale as $n$ increases.
This work presents the development and testing of a high performance implementation, written in pure Python, using time constant and $O(n)$ operations, by leveraging high performance containers and a vertex comparison method. The figures below depict results of initial testing using synthetically generated lattices of triangles, squares, and hexagons with rook contiguity in black and queen contiguity in gray. These geometries were selected to control for average neighbour cardinality and average vertex count per geometry. From these initial tests, we report a significant speedup over r-tree implementations and a more modest speedup over gridded decomposition methods. In addition to scaling linearly, while existing methods scale quadratically, this method is also more memory efficient. Ongoing work is focusing on testing using randomly distributed data, and U.S. Census data, the application of parallelization techniques to test further performance improvements, and the use of fuzzy operator to account for spatial error .
