As of 9:26 local time, we received the all-clear from REDI Net on their fiber repair. All of our services were back online yesterday at 12:18 local time. Thank you for your patience!
We just checked in with REDI Net and they are saying another couple hours on the fiber repair.
Here’s a quick summary:
At around 12:30PM today a truck damaged fiber near the China Kitchen in Española, cutting off REDI Net’s access to the outside, and cutting us off from the southern part of our network (White Rock and Pojoaque).
We have another upstream connection, but it is currently maxed out, because we split traffic between REDI Net and the other connection. As of about an hour ago, peak traffic time, the remaining connection became saturated.
Once the fiber is repaired, customers to the south should have service restored. Peak time is almost over at this point, so we anticipate any bandwidth issues caused by this outage should resolve soon.
Thank you for your patience. While this sort of thing is out of our control, we were just meeting with some of the REDI Net staff today to discuss making the southern parts of the network more redundant in case of something like this.
PS. YouTube was down worldwide for a few hours but seems to be working now.
There is currently a REDI-Net outage affecting clients in Pojoaque and causing brief slowdowns of our service. With the exception of those in Pojoaque, service should be online. We do not currently have an ETR, please check back for updates.
Today we completed a significant router upgrade affecting many parts of the Española area, and put in infrastructure to boost throughput further in the Sombrillo area. The Sombrillo upgrades will be completed as soon as we receive some backordered equipment, but we hope some of the new equipment there will boost speeds within the next week for those on the eastern part of Sombrillo road.
With the upgrades today we have found a new bottleneck that we plan to address by purchasing more bandwidth for the affected area effective October 1st at the latest. This will allow us to take full advantage of today’s upgrades.
Thanks for your patience as we continue to grow and upgrade our network. We are committed to removing all bottlenecks on our network as we find them and providing the best possible Internet service in the area.
Today, Thursday September 20th there will be a brief outage as we work to replace and upgrade some infrastructure at the HC tower location. This outage affects people in Espanola south of Walmart and all clients in Sombrillo. The outage will start around 3PM and last approximately 15 minutes.
We apologize for the short notice and thank you for your patience.
Power has been restored to our site in Española, all links have returned to normal.
A transformer feeding power to one of our hub sites has failed, bringing down communications from one of our sites and forcing three tower locations to backup connections. JMEC is on site and currently working on repairing the power situation. There is not yet an estimated time of resolution, but as soon as we know more we will provide an update.
This affects all clients from the south edge of Espanola through Alcalde.
There is a small power outage in Española affecting our customers in the Las Lomas apartments and the area of the Riverside/76 Intersection. At the moment none of our tower or core equipment is affected and service should be restored to customers once their power comes back on.
A small, inexpensive UPS like this will keep your Internet connection going during a power outage, for $40. :)
We are making changes to the network in cooperation with upstream provider REDI Net to improve redundancy. These changes will begin at 5am on August 14th. It is possible it may disrupt all customers, although we do not anticipate any noticeable effects on traffic.
Today Thursday July 26th, we are replacing the backhaul to the MD tower with a licensed link. The tower will drop offline at around 9:45 AM and will be down intermittently until approximately 11:45 AM. This outage only affects a small handful of clients.
--Black Mesa Wireless
There is an extended power outage in southern Espanola affecting several of our locations. It has gone long enough to run down some of our battery backups. We are waiting for Jemez Co-op to restore power, at which time service will be restored.
We were informed the outage was due to a failing battery backup and has been resolved. All services should now be functioning normally.
One of our Espanola towers has lost its fiber feed from REDI Net. REDI Net Network Operations is investigating the issue. In the mean time, the backup link is functioning, but some users in Espanola and western Alcalde will see reduced speeds until the issue is resolved.
A widespread power outage is affecting most of Espanola, Alcalde, Dixon, Velarde, and Chamita. Our network core and most of our towers are still running but many customers will have an internet service outage until Jemez Co-op restores power.
See http://184.108.40.206:7576/ for the Jemez Co-op Outage Map.
We are working with REDI Net to resolve the issue that caused the system-wide outage last week. They are ready to apply their fixes and we have scheduled a service window for 07:00 - 07:30 on June 26th. We are hopeful this will not cause any further downtime, but it is a possibility.
We are currently experiencing a complete network outage as of approximately 8:00 AM today. We are working with REDI-Net in an attempt to determine what and where the problem is. We do not currently have an estimated resolution time, but check back for more updates.
-Black Mesa Wireless Team
We are performing some software and equipment upgrades on the Rio Arriba Health Commons tower site this morning, which will cause intermittent brief outages for customers on that tower. This affects primarily portions of southern and western Española, including Las Lomas apartments.
We are moving the core router, which handles all traffic for the entire network, to a larger battery backup tomorrow, June 19th at 5:30AM. This should only require 5 minutes of downtime but we are leaving a 30 minute window to deal with any possible issues that may come up. This will affect ALL CUSTOMERS.
We are having some problems with a router locking up in Velarde, affecting customers in Velarde and the north end of Alcalde. We are actively troubleshooting this issue and it may require a hardware replacement.
There appears to be a widespread power outage in the southern portion of Espanola, near the Dreamcatcher area and in to La Mesilla. Clients connecting to the tower near Santa Cruz may experience service interruptions due to extended power loss.
For more information, visit http://www.jemezcoop.org/ and view their outage map.